First, this event is unique this go-around. Joe Colantonio is helping to host this event. Some of you may be familiar with Joe via Test Guild. Since the conference is virtual this time, having Joe help with running the event is a definite plus.
One of the changes this year is that they are using Slido to ask questions during the sessions. This will allow the presenters to answer the questions on the screen. There will also be virtual events happening during lunch. I intend to take part in one of those so expect me to see me doing so then :). Tomorrow, we will be recording an episode of "The Testing Show" from the "show floor".
Philip Lew is starting the event. He is reminding us of some of the questions that came up back in January. What would this conference look like? Suffice it to say, nothing like what it looks like today. Still, we are making the best of the situation. One of the benefits of the conference being virtual is that speakers can take part from anywhere. One of the keynote speakers is presenting from Australia. We do not have the poster papers this time but there are many technical briefs that are being presented.
Jason Arbon is the first keynote speaker. He is speaking about TestOps. The idea behind TestOps is to shake up the way that testing is being performed. It also allows for a way to have testing be present. Jason focuses on testing on a large scale and is currently with test.ai.
TestOps is the idea of operationalizing testing. the goal is to automate as many things, make as reliable as possible, and scale as much as possible. Testing is a unique area in that automation is often difficult. Tests can be flaky. Often, decisions focus on shipping even if the testing is not complete. Unit testing can't answer everything. Many areas are resistant to running end to end tests. Jason used the example of a car purchasing application that was frustrating to use. Its functionality was not working well. Yet, this application shipped as is and considered usable. It's likely that this application passed all internal unit tests. Still, a product can pass all tests and still be unusable.
Tests should be short-lived. They should focus on the simple rather than the complex. They should be capable of running fast. Consider we have ten tests that we have to perform and one of those tests is signing in. What if signing in is actually difficult to do because of security implementations? How do you test in parallel with one sign in? The answer is you drop sign-in as a necessary test and focus on other areas that don't need sign-in. Heresy, you say? There may be other ways to do the same thing. There may be direct calls that can run via API. There may be individual authentical calls. Again, think simple where possible.
Maintaining attributes and states can be difficult. Avoid framework tweaks that may seem to be effective in the short term but become unreliable later on. Additionally, try to avoid the need to create an elaborate dashboard to display results. It seems counterintuitive but instead focuses on sending notifications if a specific problem is found. alert the right people at the right time in real-time.
Maintaining attributes and states can be difficult. Avoid framework tweaks. They may seem to be effective in the short term but become unreliable later on. Try to avoid the need to create an elaborate dashboard to display results. Focus on sending notifications for a specific problem. Alert the right people at the right time.
The more frequent the updates, the less likely complete testing will happen. It's tempting to think that eliminating testers is an answer. Instead, there needs to be a focus on good, robust end-to-end tests. Also, those tests need review and consideration by real testers. It's not enough for developers alone to do the testing. That's not to say that they can't do it but they need to focus on that testing as its own discipline.
Jason is a proponent of AI, which comes as no surprise considering test.ai and their focus. Frameworks and test applications have limitations that force hard-coded tests and methods. In a TestOps world, if the tests are not reliable, they will not execute. Many testers get thrown into automation without previous development experience. Putting a new developer into the space of automated testing can be a recipe for frustration. Not to say that new developers can't write tests. They can, and should, but it shouldn't be their first programming assignment. These tests need to be robust, they need to have the ability to be abstract, and they need to be adaptable. That will take some experience that the average tester does not yet have.
TestOps is happening, it's real, and it's not just an abstract goal. Test what you can, as cheaply and reliably as you can. This isn't abstract, it's a reality here and now and we will benefit if we adopt it.
1 comment:
The talk should be titled TestAutomationOps - it has nothing to do with testing. It's still useful.
Post a Comment