We are Underway: Building on Success, Beyond the Obvious (#PNSQC2021 Live Blog)

 

It's exciting to be back at the Pacific Northwest Software Quality Conference (PNSQC). Granted, we are remote and virtual for a second year but I have enjoyed being a participant at this conference for the past ten-plus years. At this point, I think I have attended and presented at PNSQC more than at any other conference. I've gone the whole route from writing papers, presenting posters, giving full talks, being an invited speaker, and being a finalist for best presentation. I've learned a lot here and I look forward to continued participation. Yes, I have a talk I will be presenting here as well, but for obvious reasons, I can't live blog my own talk ;).

Monday is starting off with a keynote being given by Eric Van Veenendaal discussing "Building on Success, Beyond the Obvious". It's important to start with a simple question, "What is success? What happens and what makes a difference?" In other words, what can we do to actually be successful without necessarily being "THE BEST". 

Eric pivots here and takes on the "No More Testers" rhetoric. Will we indeed see a day where there will be no more testers? I'd say that the real answer is "it depends". I think testing will always be relevant and testing will need to be performed by skilled people. I also do not believe that testing has to be performed by a dedicated tester. Fortunately, many of us who have come up in the trenches as software testers are also not only doing testing. We may do a lot of it, but odds are we are involved in many other initiatives that help the success of the company. 

Having been in or around testing for thirty years now, I can say that the testing landscape and the focus of my efforts have changed many times over those three decades. The basic methods I learned in many ways are still relevant but the overall approach and HOW I test, much less WHAT I test, has changed dramatically. 

An interesting comparison is made with the idea that Agile adoption has grown but the achievement of the benefits from it looks to be elusive. Doing Agile has benefits but it does not guarantee better quality software. 

As a teacher of the BBST courses, I realize that there is a large gap between what we say and espouse testers to focus their efforts on and what actually gets focused on in the real world. Even when I train and work with people to use these skills and techniques, often they fall apart in a real-world application. I'm a Senior Automation Engineer by title but I'll say that my day-to-day testing efforts are still a large majority manual and exploration based. More to the point, I get to do a number of other things that go beyond just handling testing initiatives. this may sound bleak and yet, we release, we fulfill obligations, we get software out to our customers, and they make it work for their purposes. Ideally? No, certainly not. Effectively and usably? Yes. Thus a lot of what we do and focus on answers to the various Agile rituals, truth is we make do and make it work to be effective. It's not a perfect process at all but it does work, for some definition of "work" ;).

It's one thing to "do the things that make the difference" and that's really the key here. Getting to that place can be difficult because there are a lot of turfs that are often defended vigorously. In many cases, what we do is not nearly as important as how and why we do it.  

Set Clear Priorities and Understand The Risks

In many cases, testing comes down to communicating the risks. What would happen if the worst-case scenario occurred? How would we effectively approach our testing if we placed a serious focus on the risk assessment and taking a risk-based approach to testing? This doesn't necessarily mean "more time testing" but "testing the areas that would have the greatest risk were they to go wrong". This means we need to be focused on setting priorities related to risk identification, analysis, and mitigation. Additionally, how many areas do we have specific control over, and what areas are peripheral to us yet still important? Will we be dealing with things that would have a high impact if they went sideways? How likely is that sideways issue to happen? The better we can get a handle on these two values, the better we will be at developing our risk-based test approach.

I like what Erik says here to the effect that a test plan should reside on a single page where possible. We should have a test approach that deals with where to focus and determine what matters the most and what may not matter anywhere near as much.

Review Processes and Methods to Determine what is Actually Working/Important

Shift-left is a popular term these days and a part of that is looking at ways to be involved and focused on issues that are discovered and how they may be mitigated in the future. Part of the shift-left approach is increasing visibility on testing efforts as soon as possible in the process. Reviewing past efforts and learning what we can do to be effective EARLIER  in the process is a helpful approach. To be clear, this doesn't mean review everything all the time. That's all we would be doing. The key to effective reviews is to look at specific areas, scope them so as to not be overwhelming, and work on them a little bit at a time.   

Use More Effective Unit Testing to Help Find Issues Earlier

This may seem obvious but this is not to say "do Unit Testing". Hopefully, most organizations are doing Unit Testing as an active practice. This is taking a development methodology and thinking about how and where to use it to help with the development process. The potential issue here is that many developers have not been trained in testing techniques or skills. Thus, while there may not be a need for a dedicated tester in many circumstances, that means that there needs to be a developer(s) who are well versed and experienced in testing techniques and can bring that back to their software development. To say that developers don't know how to test is ludicrous, as many developers are very good at testing. The challenge is many other developers are not. How do we bring them up to speed?

Introduce Test Design to the Development Process

One area where testers can be helpful early on is identifying test conditions and test situations early on. In many cases, testers will not be able to address every area upfront or early in a development cycle. Over time, though, we will develop familiarity and we will have an understanding of the areas we have already tested and worked with, reviewed, and evaluated with the team. This will allow us to consider testing, testability, and risk areas early in the development process and ultimately make it possible for us to address testing in an effective manner. Exploratory testing is important in the early learning stages. It's important at many other times but it can be make-or-break in these early phases of development. this is the time where we can address areas like testability, accessibility, inclusive and responsive design.

Build Experienced and Skilled Testers, Whoever Those "Testers" Happen to Be

When we talk about training the testers, realize that there may be many different people testing at any given time. Dedicated testers, sure, but also developers, support engineers, operations people, stakeholders, customers (indirectly but they will test whether we like it or not). No one is going to be able to be excellent at everything but everyone can and will develop broad skills and knowledge that will be helpful to the process and projects. This also includes a variety of soft skills. Testing and bug hunting is important but just as important is advocacy and developing the ability to present cases on behalf of others and speak to the risks and challenges and what they may face. Additionally, think about the members of teams and what they are doing. If the attitude of "no more dedicated testers" is a potential reality, as what the plan is with those people above and beyond testing. Do we plan on having them add value in development? In Operations? Assisting support? Each organization will have different goals and needs but again, investing in people will net great results down the road.