The Testing Team Have Requirements Too with Moira Tuffs (#PNSQC2021 Live Blog)

 

All too often we see requirements being used for development, customer feature needs, business requirements, etc. To use Moira's company example: 

"speed improvements of 30%" 

"user should see plain English error messages." 

A big question right here... how do we actually confirm/verify that we can actually do/meet these requirements? In short, this hearkens back to one of my previous presentations... "Is this Testable?" 

We expect a 30% speed enhancement... how are we determining the fact that this 30% improvement has actually happened. 30% faster than what, exactly?

Basically, we as testers need to know what the baseline is and how to measure it. We need to have the initial hypotheses and understand them (it seems hypotheses is going to be my favorite word this conference (LOL!) ). 

I can feel this deeply in my own organization. My current test role has me being the "newbie" of the team. I've worked with them for over 18 months now and yet it seems in every story there's some new area that I need to know about that I have never even heard of. My team has been together and working on our area of expertise (data transformations) for more than a decade. Yes, my team members outside of myself and our scrum master have a decade-plus of working with each other. That means there is a tremendous amount of explicit and implicit knowledge locked up in those brains that get taken for granted and I am the one who has to try to figure out what piece of implicit knowledge is missing. 

This is interesting as Moira is hitting a lot of the areas that I remember from doing my "Is This Testable?" talk. I am in full agreement that log files can be an absolute gem in helping define and highlight implicit requirements or areas that we should be paying attention to. It may take some time to get to understand what information is relevant and interesting, and there may be many log files to have to interact with. Here's where I recommend firing up your favorite screen multiplexer and cut up a screen window to tail a variety of log files. This gives us a chance to see if we are actually seeing the data that is relevant given the transaction at hand. 

Moira is using an example that has both a software and a hardware focus. The software can be squishy but if the hardware doesn't work right, there are real issues that may need a more direct interaction. It might be as simple as clearing a nozzle for the material to flow through (this is an example using a 3D printer as the hardware). It may be different where the software is specifically the differentiator. This is something I often deal with in my home studio and with my audio interfaces. I have multi-input systems with multiple outputs that can be routed. Thus for me to be effective, I have to have a firm understanding of the control software that allows me to create signal routes on the fly. Since these input and output routing values are not going to always be in place, there's a lot of interaction, setup, and teardown to be able to route effectively for different sessions and purposes. I'm mentioning this to mainly say that I feel for my testing kin over at Focusrite (and thank you for what you all do to make this less odious than it often can be :) ).

Product Ecology and Context Analysis: #PNSQC Live Blog

Ruud Cox is an interesting dude who has done some interesting stuff. To lead off his talk, he described a product that he was involved with testing. A deep brain stimulator is effectively a pacemaker for the brain. Ruud described the purpose of the device, the requirements, and issues that have come to light during the procedures. I've worked on what I consider to be some interesting projects, but nothing remotely like this.

Ruud was responsible for organizing testing for this project, and he said he immediately realized that the context for this project was difficult to pin down. In his words, the context is a blur. However, by stepping back and addressing who the users of the product are (both the doctors and medical staff performing the procedures and the individuals who are having the procedures performed).

One tool that Ruud found to be helpful was to create a context diagram, and in the process, he was able to sketch out all of the people, issues, and cases where the context was applicable, and that it was somewhat fluid. This is important because as you build out and learn about what people value, you start to see that context itself shifts and that the needs of one user or stakeholder may differ from, or even conflict with, another user.

Patterns and giving those patterns meaning are individual and unique. Ruud points out that, as he as making his context diagram, he was starting to see patterns and areas where certain domains were in control and other domains that had different rules and needs. Our brains are "belief" engines", meaning we often believe what we see, and we interpret what we see based on our mental model of the world. Therefore, the more actively we work with diagramming context, the more we understand the interactions.

Ruud refers to these interactions and the way that we perceive them as "Product Ecologies". As an example, he showed how a person has been asked if they can make an apple pie. Once the person says yes, they then look at the factors they need to consider to make the pie. The PRoduct Ecology considers where the apples come from, what other ingredients are needed and where they come from, the tools and necessary methods of preparing and combining to create an apple pie suitable for eating. In short, there' a lot that goes into making an apple pie.

Areas that appear in a Context analysis need to be gathered and looked at in regards to their respective domains. Ruud has the following  approach to gathering factors:

• Study artifacts which are available
• Interview Domain experts
• Explore existing products
• Hang out at the coffee machine (communicate with people, basically)
• Make site visits

Another way to get insights on your context is to examine the value chain. Who benefits from the arrangements and in what way. Who supplies the elements necessary to create the value? Who are the players? What do they need? How do they affect the way that a product is shaped and developed over its life?

User scenarios try to diagram the various ways that a user might interact with a product. The more users we have, the more unique the scenarios we will accumulate and the more likely that we will discover areas that are complementary and contradictory. Ruud showed some examples of a car park with lighting that was meant to come on when being used and go dark when not. As he as diagramming out the possible options, he realized that the plane and the angle of the pavement had an effect on the way that the lights were aligned, how they turned on or off, or even if they turned on or off.

Currently, Ruud is working with ASML, which creates tiny circuit elements on chips. One of the factors he deals with is that, to create a wafer in a scanner and fabricator, it can take months to produce a single wafer. Testing a machine like this must be a beast! Gathering factors and requirements likewise is also a best, but it can be done once the key customers have been identified.