First of all, thank you to everyone who came to my talk about "The Dos and Don'ts of Accessibility". Seriously, it's been a great feeling to know that a place that has been so pivotal in the lives and futures of deaf and hard of hearing individuals (Gallaudet University) is the setting for my talk. How cool is that :)? I'll sum up that talk at a later date but for right now, let's go to Paul Grizzaffi and talk about the "myths about the myths of automation" (and no that's not a typo, that's the literal title :) )
There are a lot of myths when it comes to automation but are there now myths around the myths? According to Paul, yes, there is.
Is Record and Playback bad? No, not necessarily. It can in fact be a very useful tool to use in a stable environment where the front end doesn't change. It's not so good for actively under development systems, especially if the front end is in flux/development.
Do you have to be a programmer to use automation tools? No, not necessarily but it will certainly help if you have some understanding of programming or have access to a programmer that you can work with.
Does Automation Come from Test Cases? Not entirely. It can certainly provide value but it doesn't necessarily make sense to take all of your manual test cases and automate them. For a few valuable workflows, then yes, but if doing so will have you repeating yourself and adding time to repetitive steps, then it may not be the best use of your time. Your test cases can be helpful in this process and they can inform what you do but don't just automate everything for the sake of automating everything.
Does Automation Solve all testing Problems? Come on (LOL!). Yeah, that was an easy one but it can often be seen that running a lot of tests quickly can seem to be a high-value use of time, where it can instead just be doing a lot of busywork that looks like a lot is being done but not much that is meaningful.
Will Automation Find all of your bugs? NO, 1,000 times NO!!! It can show you if a code change now renders an older test a failure, which you can then examine afterward. It can help you with more coverage because now you might be able to make a matrix that will cover a lot of options and run against an orthogonal array. That can be useful and provide a lot of test case coverage but that's not the same thing as finding all of the issues.
Can we achieve 100% automation? Nope, at least not in the meaningful sense of 100% automation. You can certainly have a lot of workflows and matrices covered, and machines are much faster than humans. However, there will always be more workflows than you can automate. We're not there yet in regards to being able to automate 100% of the things. Even if we could, it will likely not be a good overall use of our time to automate all of the things. Automate the most important things? Sure.
Is There One Tool To Rule Them All? Absolutely not. Yes, shared code can be a benefit, and yes, buying many licenses can help unify a team or teams but it's highly unlikely that a single tool is going to answer everything for everyone. That's not to say that there isn't value on a standard baseline. We use a number of libraries and functions that allow us to test across a variety of products but no one tool covers everything.
Plain and simple, as in all things, context matters and no two teams are the same. Look at the myths you may be carrying and see how they measure up to the reality of your organization.
No comments:
Post a Comment