Mike Lyles is covering something I like to talk and present on, so I felt it would be good to get another perspective. Visual Testing is important because, in many ways, we need to use our eyes to validate something an automated test cannot, and yet, our own eyes are untrustworthy. We fill in a lot of the details that we expect to see. We are remarkably good at inferring what we think we should be seeing, so much so that we can miss what would be obvious (or at least, what we think should be obvious on closer inspection).
Assumptions come from one place; the brain will sacrifice facts for efficiency. This is why we can be tricked into thinking we are seeing one thing when the full image shows we are looking at something else. On the bright side, the brain is constantly evolving (thank goodness the "your brain is locked at 25" idea has been debunked). More relevant is that the brain does not want to pay attention to boring things. We get conned into attempting to multi-task because we crave novelty, yet the brain can't really multi-task, so we make shortcuts to help us deal with what's in front of us. In short, we become more fallible the more novelty we crave. Additionally, the more stress we deal with, the more pronounced this cycle becomes.
One of the things I often suggest, and I learned this from James Lyndsay a few years back, is the need to repeatedly focus and defocus. We need to focus to see patterns, but we can often become so fixated that we miss other patterns that are clearly in front of us. Fans of the game SET are probably familiar with this, especially when the pattern is picked by someone else. With stress, the ability to focus on what's in front of you diminishes, but if you practice working with the stress, you can overcome your diminishing faculties.
Other senses can help us remember things, but what we see is usually more memorable than anything we experience in any other way. Mike gave two pieces of paper to two participants, telling the same story, but one was just words, and another had pictures. The person with the annotated pictures remembered a lot more of the details of the story.
People are naturally curious, and little children especially so. Over time, we tend to tamp down on that natural curiosity in favor of efficiency and performance. Good testers will work to try to awaken that natural curiosity, and they will try to open themselves up to approaching problems from a variety of angles.
We often approach systems with a pre-conceived notion of what they should do and where they should be placed. There's a comfort in standardization, but that can also led us astray when an item is designed intentionally to thwart that expectation. In many cases, we interact with the device the way we expect to, not how it is actually configured. While I may say it is overkill to look at every device with fresh eyes, it does make for an interesting experiment to try to put aside the expectations we have and try to look at it as a brand new object.
One interesting example Mike used was to pull out a jar of gum balls and ask us how many there were. Some people set low numbers (I guessed three hundred), some guessed high (close to 1000) but as we recorded more guesses and then took the average, the average was within ten of the actual count. In short. a collection of observations and comments gets us pretty close to the reality, whereas one of us might be wildly off, based on our limited visibility and understanding. In some cases, the crowd sees better than the individual.
Good thing to remember is that what we think we are looking at may not be at all what we are looking at. Plan accordingly ;).
No comments:
Post a Comment