Amplifying Agile Quality Practices with Selena Delesie (PNSQC)

I had the opportunity and privilege to introduce Selena Delesie on the program today. It was fun to reminisce a bit because Selena and I were both in the same Foundations class for Black Box Software Testing all the way back in 2010. We also both served on the Board of Directors for AST, so we had a lot of memories and fun/challenging things to deal with over the years. Thus, it was a pleasure to introduce Selena as our first Keynote speaker. Also, part of her talk was discussed on a recent The Testing Show podcast, so if you want a sample, you can listen to that :).

The tool that Selena and Janet Gregory put together is called the Quality Practices Assessment Model (QPAM). The idea behind this is that there are ways to identify potential breakdowns in the quality of our work. Areas we should consider are:

• Feedback Loops
• Culture
• Learning and Improvement
• Development Approach
• Quality and Test Ownership
• Testing Breadth
• Code Quality and Technical Debt
• Test Automation and Tools
• Deployment Pipeline
• Defect Management

The fascinating thing is the order and how these are identified and examined. Selena makes the case that the order in which these are presented and examined is important and by examining them in this order or weighting, the best chances for overall and lasting improvement are possible. Yes, Defect management is important but it will be less effective if more weight is not given to the previously mentioned items.

A key aspect to this is that quality is not just a technical issue, it's also a social issue and it should not be dealt with in isolation. Selena introduces us to a group code-named "Team Venus" and identifies many of the issues they are facing and where those issues fall on the ten quality aspects. The key element is that each area is looked at holistically and in conjunction with the other areas, not in isolation. As anyone familiar with GeneralSystems Thinking can tell you, there is no such thing as a standalone and isolated change, any modifications made will have ripple effect. It's also critical to realize that a process alone is meaningless if the overall values are not solid or agreed upon. 

In the ten quality aspects that Selena referenced, there are four quadrants/dimensions to consider:

• Beginning
• Unifying
• Practicing
• Innovating

What I like about considering these as quadrants is the fact that these areas are not separate from each other but they are dependent on each other. Some areas of the ten practices will be closer aligned with a particular dimension. Additionally, it's common for teams to spend more time in a given quadrant/dimension for the ten areas than others. I like the diagram Selena uses that looks like a spider web. The center of the web means that that is an informational or foundational level, and the farther out from the center, the greater the expertise and experience. Ideally, of course, all of the aspects should be on the outer rim of the spider web but in practice, there will be color splotches in all four of the dimensions. That is normal and should not be discouraged, especially since each new team member will typically need to start from zero.

For those interested, the book and full model example for Team Venus is available in "Assessing Agile Quality Practices with QPAM" so if you want to learn more, go there.

I’m A BA Girl In Agile World: An #InflectraCON Live Blog

Mindy Bohannon leads off with the idea that the Business Analyst (BA) can do many things, including own products, help with design, and also help with testing. Being a BA allows for a lot of leeway as to what they actually do and when they actually do it.

I've often been curious as to what a BA actually does. They can bring a number of things to the table they can be:

- the Proxy Product Owner

- the User Experience Expert

- the Tester (or one of them)

- the Scrum Master

As well as the more typical roles of:

- Data Analyst

- System Analyst

- Process Analyst

Okay, that's neat and all but for those of us in the back... what is Business Analysis in the first place? BA defines what the need is and then recommends solutions that deliver actual value. They also define and validate the solutions that meet business needs, goals, and objectives. 

The core focus of a BA? Communication, working with stakeholders, Facilitating conversations and work needs, prioritizing those needs, and defining/explaining the requirements and how they are implemented. An important skill to be emphasized is the ability to have a flexible mindset. The requirements as defined originally may change, and they may need to change rapidly and frequently. BA's will need to continually assess and adjust based on what they learned, as well as the fact that they may need to operate on more of a Just in Time development model for requirements. In short, Agile requires flexibility and a willingness to work on things up to and including the last responsible moment (and maybe the last irresponsible moment :) ).

The key to note here is that the BA is a much more versatile role than creating requirements. If we have them, leverage them. 

How to Nudge Your Way Through Agile Testing: An #InflectraCON LIve Blog

 I confess the title alone convinced me to come to this talk. I've never met or interacted wth Ard Kramer before. Ard calls himself a "Qualisopher" (Quality Philosopher, nerdy to a fault and I like him already ;) ). Ard starts out with an interesting question... "Do you drive better than the average driver?" When we ask such a question, we have to think "what is average?" What does that represent? Are we more favorable for ourselves or are we less favorable? In other words, do we perceive ourself as being better than others, at least in an average sense?

We have a lot of biases, as the question helps illustrate. We have our own biases and those biases are actually things that we can leverage to our advantage. Marketing uses biases to help sell all the time. Maybe we can use that knowledge to help us. This is the key to where "nudging" comes in. By leveraging the biases of people, we can encourage behaviors. By encouraging behaviors, we effectively give them a nudge. An example that was used in the talk was the idea that by making something fun (in this case, a waste receptacle was wired for found so that any time something was thrown into the waste bin, a noise of a falling object (in a cartoonish manner) was played back. This sound was amusing, so for many people, it was fun to use that waste bin. Over time, that bin had more garbage thrown away in it over other receptacles that didn't have the sound wired for it. By making it a little more fun, the municipality changed/nudged the behavior of its residents.

There are two main systems that people put into play when we are working on things, one is the sense of the "Doer" while the other is that of the "Planner". While most of us like to think we are capable of doing the second, we are actually much more focused on the former. Both have their benefits and advantages but both are different from one another. The question then comes down to "how can we use these models to help us "nudge" people in desired directions?" In the sense of testing, how do we feel about the product we use? IS it a product I can stand behind? If so, then it is likely that we can nudge people to help come along for the ride we hope to have people go on. 

An effect that is used is "The default option". In many cases, this is an area that few people explore. If the default is either too low or too high, there is a likelihood that people will not be happy with the results. It is safe to say, though, that if something requires a lot of interactions and confirmations, the abandonment rate to accomplish a task/create an order increases considerably. Thus, a reasonable set of defaults is helpful but the defaults need to be explained in a way that will make sense and be appreciated by the user(s). 

Another effect that is helpful is consistency. When people do things and are encouraged to do them, they often get a greater commitment. Why? Because when people give themselves the ability to accomplish tasks for and by themselves, it solidifies ownership of the process and thus greater consistency in accomplishing and succeeding in the task. 

Additionally, things we are asked to sign off on tend to get a greater commitment. Why? We are not anonymous at this point, we are now putting out there our own desires and focuses. By signing our name to a process, we own it, we own the outcome, and that either gives us great pride or great pause. Either way, we tend to be focused on what we are doing because we are known entities doing so.

An interesting approach that I had not heard before is "The Zeigarnik Effect". The idea here is to leave things open or unresolved because we are more likely to remember things that are not completed. Once we have completed something, well, completely, we tend to offload the learning we have done. Not completely but somewhat and then it is hard to recall where everything is. By ending a given day's efforts with a question allows us to hang on to what we have learned.

Another clever approach is to trigger an unconscious behavior. One example was the idea that trains that had quiet car designations struggled to encourage people to honor the principle of the quiet car. What they did to solve the problem was paint the cars to suggest/represent the users were in a library. What's the typical behavior of people in libraries? They tend to treat it as a space to be silent, or at least quiet. By painting the cars to look like actual libraries, they helped set people's behaviors as to how they would react and interact. Frankly, I think that is clever :).

There are lots of methodologies we can use to allow us to do the things we need to and what we expect others to do. Nudges are just that, they are not extreme, they are not unpleasant, and they are often simple things that can help guide people to move just a little bit one way and accomplish tasks in a better way.

Is There Room for Test Strategy in Agile? (#PNSQC2021 Live Blog)

Suffice it to say, today is going by quickly. We are already up to the lunchtime keynote and immediately following, it's go-time for me! However, before that, we get to hear Mirjana Kolarov talk about "Is there Room for Test Strategy in Agile? It's funny to hear how Mirjana, like so many people in software testing, came to it from an oblique angle. Are there any testers that actually chose it as a first career ;)?

So let's start with the base question... what do I think when  I hear the words "Test Strategy"? 

Honestly, I think of pre-year 2000 testing projects with the need for large documents that nobody actually reads to sign off on testing in test pans that no one actually does to meet standards that no one actually checks. 

Sounds cynical, huh? 

Well, that heavy-duty documentation by its very existence made testing more difficult than it needed to be Let's be clear, though, I'm complaining about the documentation and the heaviness of it by some arbitrary mandate. A test strategy on its own I'm actually very much in the "let's do this" camp.

 

So what is the actual benefit of having a test strategy? It's not to meet that arbitrary "fill in the template" need of middle managers. Instead, it should be looked at as:

• What will guide our testing? 
• Do we have a common understanding of our testing processes? 
• Do we know what our deliverables are? 
• What are the expectations of the given test strategy?

Like Mirjana, I would encourage limited test strategies for given projects and areas of influence. rather than have some overall encompassing test strategy for an organization or a project, how about we focus on a given project or instead have a strategy for a specific story? Mirjana points to a larger overall strategy, so if I were to gather multiple test strategies into one document, then yes, I could imagine them being the same size (or roughly the same). 

An Example Test Strategy outline 

Mirjana points out that writing a test strategy in isolation is a task doomed to failure. It needs to be informed by the needs and the goals of the group, the project, and the over-arching organization. Having a single test strategy for an organization may or may not make sense. In some cases, a large test strategy is irrelevant to incremental Agile development. That's not to say there is no need for a strategy but instead, having a modular test strategy that can be addressed for each area both as needed and as defined can come into play. Again, having twenty different test strategies for multiple areas of interest may be a little different compared to one large document in volume, time, and effort but if a test strategy for a particular area is one or two pages, it's much more likely that those pages will actually be read and reviewed. 

Additionally, there are other ways to represent a testing strategy beyond a standard document, table, or spreadsheet. Mirjana makes the case that simple mind maps may be able to communicate the same level of information that a more dense document would. 

Ultimately, the level of trust and understanding of the team will determine how these test strategies are both communicated and documented. Information Radiators can be as simple as a page in a wiki or post-it notes on a whiteboard. The key takeaway, at least to me, is not the formality of the document but the ability to communicate it effectively and quickly. The more closely knit and actively the team works, the less formal or voluminous the test strategy or any documentation needs to be. The more distributed the team, the more necessary those same strategies and guidelines be codified and visible (they still need not be overly long or arbitrarily wordy). Overall, are we trying to create an effective method for testing that we can communicate to the entire organization? If the goal is to have testing happening all the time, everywhere, in every aspect, the more people who understand and know the testing goals, the better :).

PNSQC 2020 Live Blog: Towards A Culturally Inclusive Software Quality with Jack McDowell & Ying Ki Kwong

It's been interesting to see how we have seen the world adapt over the past thirty years that I have been in the software testing industry. In some ways, it's a very different world and yet at the same time, not much has changed. When I came into the testing world, there was a need for understanding requirements, there was a need to know how to test and there was a need to execute a certain number of tests, manually if we must but automated would be spectacular.

What I just described was 1991. Does it really sound that much different than 2020? Let's take a look at some other factors. Who is doing the work? For me, I remember thinking about this as I was working for Cisco Systems and I was hearing how few women or PoC were actively involved in tech. I frequently wondered what they were talking about as I was literally surrounded by women as peers and women as managers, as well as a rather diverse team as related to cultural and ethnic background. It would take me several years and watch the company grow to have that lens be changed and see the point. Where am I going with this? It's sometimes hard to see the issues and challenges we face when we think that the world being described doesn't match our experiences. I made the mistake of thinking the early days of Cisco Systems were indicative of the world of tech as a whole. 

Again, interesting but Michael, what are you going on about? Well, it's key to understanding Jack and Ying Ki's talk (and Jack is the one delivering it but since Ying Ki helped write the paper, I'm giving him credit, too ;) ). In any event, the point is that we see the world through our lens, and that lens may be accurate or not but we won't know it is accurate if we don't have other points of reference. With this idea, we should also be asking "what does quality actually mean?" It means whatever the team at large decides it means, and then we radiate out from there. However, we are missing a bit of the point that quality is going to be skewed if we don't seek to pay attention to what others think that quality is.  

An idea that Jack presents is that we not look at trees as a metaphor for meaning but instead look at rhizomes. A rhizome is a plant that spreads out a root system and buds, vines, etc grow out at a variety of points. we might look at these as being multiple plants but in truth, it's the same plant. 

Ha! Communities of meaning using Accessibility. We're in my court now (LOL!). Well, not really but it is interesting to see what examples for accessibility are used and what actually counts. Jack is correct that accessibility often looks at disabilities as a monolith when in reality, there are a variety of unique disabilities that require very different methods of accommodation. I'm all too familiar with the approach of "load up a screen reader, walk the application and call it a day". Yes, I've been there and so have a lot of others. The fact is, that approach leaves a *LOT* of things out of the mix or even giving consideration. Sight issues are not the same as hearing issues, and they have little to do with mobility issues, and those have little to do with cognitive issues, and on and on.

So Standards are often included in these quality questions. Standards can often be helpful but standards also tend to take a prescribed way to identify what makes for software quality. yet the question remains, does a standard take into account cultural differences of other approaches? Jack is arguing no and I'd say I agree with him. Jack just said that Agile Software Development is pretty much Anglo-centric. Yang Ki refutes that the way that agile software development works in the USA would not be culturally acceptable in Honk Kong, for example. The idea that individual engineers would be able to express their opinions or desires about products and openly question management. It also shows that in other cultures the Agile principles can be subverted and exploited. 

Again, the question we want to consider is "how does culture affect quality?" The answer is there is really no monoculture in all of this, there is a mixture of cultures that blend together and often help define these communities of meaning. How this is actually accomplished may well be a long and involved process but it is interesting to consider other communities of meaning to address these issues.

PNSQC 2020 Live Blog: Capacity To Execute with John Cvetko

The first talk of the afternoon is being given by John Cvetko. The main point of John's talk is to look at the desire of an organization to change from a Waterfall development methodology to an Agile development methodology. for many of us, it may seem slightly odd that there are still Waterfall SDLC's out there but rest assured there certainly are, lots more than we might like to think. John focused on the considerations needed to make the changes necessary to make the move from Waterfall to agile.

The example organization had multiple release trains (36 release trains in total) and part of the initial process was to create a hybrid release train. They didn't go all in or replace every single release train. Rather, they took on the Agile transformation within a few release trains. They took an isolated and low risk approach so that no more than 20% of the projects would be affected. the Agile transition even with this focus and need was still going to be a challenge and that the implementation of Agile was going to require an entirely different way of thinking.   




This gets us to the title of the talk. The “capacity to execute” is the most important aspect, in short,"Plan your work and work your plan". More to the point, there's more need for planning to make a transition effect. Whether the approach is to make a huge sea change, or smaller waves or bubble up changes, there has to be the ability to plan and execute effectively. 

There are a variety of metrics that are looked at (an example is ISO 25010 Software Quality Standard) and which quality characteristics were chosen to determine the level of focus of the transition. 

By focusing on Scope, Schedule, Budget, and Quality, we as an organization can make a real determination as to what the transition will be able to focus on and to what extent. Just because we want to make the change doesn't mean we will be able to automatically get there just because we want to. Often the business context, the company culture, and the revenue and cash on hand will help influence those paths. 

There is often a Culture Clash that may occur when each team works to try to make these transitions. thus the intentions and plans to execute are doubly important at this time. 

Make no mistake, these are tricky and challenging transitions. It may very well prove to be worth it but it just as likely will not be easy.

 

Agile Where Agile Fears To Tread! - a #PNSQC2019 Live Blog

Thomas Cagley is a fun guy to listen to in just about any situation. He's been a guest on The Testing Show and I've listened to his own SPAMcast podcast (SPAM in this case means "Software Process and Measurement" ;) ) but this is the first time I've actually listened to him speak.

Tom opens up his conversation with a word I will not even try to write out much less pronounce but it has to do with Germany's Strict Beer Purity Law that has existed since the Middle Ages in Bavaria. He then mentions that his wife is gluten intolerant and thus, because of that she cannot drink beer. She can, however, drink something referred to as a "tea beer". Is it "beer" in the classic sense related to the German Brew Law? No. Does that render the beverage irrelevant or not enjoyable? Not in the slightest :). I can't speak for the beverage in question and the idea of a "tea beer" absolutely intrigues me (I don't drink but I do find the concept fascinating).


The key point to this is that we are back to the lower case versus upper case "[aA]gile". As Dawn Haynes suggested that we should tear down the Agile Temple (granted, those are more my words than hers), Tom is discussing the idea that we can take elements of Agile that are helpful and we can use them in the places we want to without necessarily committing to a full-blown "Religion of Agile"

The goal of "agility" is to move more nimbly, to be quicker, to not have to do such heavy lifts, and be able to get the product out to customers faster and more frequently, with less risk. News flash, you do not have to adopt every element of the Holy Eucharist of the Agile Temple (wow, sorry if that sounds a little harsh but I'm picking the words for the vividness, not for the snark... well, not maybe a little bit for the snark). Basically, we're back to the Beer Law. If it's not [ABCDEFGHIJK] then it's not Agile. Well, OK, maybe we can't do [ABCDEFGHIJK] but what if we were able to do [AEFGJK]. Do we just throw up our hands and say "nope, can't do just those, that wouldn't be Agile". Sadly, there are people who do say exactly that. I'm a fairly polite fellow and as such I will not call these people what they richly deserve to be called (hint: it rhymes with "boron").

As smaller components are implemented and benefits are seen what is likely to happen? The other elements will either fall into place as a matter of course or they can be introduced as needed and improved upon as the team gets better at what they do.

Being More Agile Without Doing Agile - a #PNSQC2019 Live Blog

Can I share a possibly unpopular opinion? I am not a fan of "Agile".

Now wait, let me clarify. I love BEING agile. Heck, who doesn't? I don't have a problem with the adjective. I have a problem with the Noun.

Also a confession. I'm here mainly because Dawn Haynes is talking. I've known Dawn for years and the irony is that I have had precious few times that I have actually been able to hear Dawn speak. Thus I consider this a perfect blend of opportunity and attitude :).

I like "little a" agility. Again, the actions and abilities. Those are all good things. They are helpful and necessary.  I like being nimble and quick where I can be.

What I have found less appealing in "Big A" Agile. Mainly because I find that when organizations try to implement "Big A" Agile, they become anything but "little a" agile.

As a software tester, I have often found that there is an afterthought when it comes to testing in Agile implementations. More times than not, what results are teams that kind of, sort of, maybe do some Agile stuff and then retrofit everything that doesn't actually feel right into a safer space.

Dawn emphasizes that the best way to achieve the goals of "Agile" is to actually "be agile". In other words, forget the process (for a moment) and focus on yourself and what you’re trying to accomplish.

A Comfort Zone is a Beautiful Place but Nothing Ever Grows There

For teams to get better, they have to be willing to go to places they don't really want to go to. There is a fear that going into the unknown will slow us down, will send us down paths we are not thrilled about going down, may not even get us to our end goal quickly. So we put a lot of emphasis on what I call "the priestly caste" and "the temple incantations". I'm not trying to be flip here, I'm saying that there are a lot of rituals that we reach for when we are not 100% sure about what we are or should be doing. As long as the rituals are met, we see comfort there, even if the rituals are adding little to no actual benefit. Are retrospectives helpful? They can be if they are acted upon. If they aren't then it is an empty ritual. Granted, it may take time and commitment to see the results of the retrospective findings and real results may not be manifest for weeks or months. Still, if we do not see that there are actual improvements coming from those retros, what is the point of doing them?

One of the interesting developments on my team related to agility and moving more quickly and effectively was to allow myself to wear whatever hat was needed at the moment. I'm not just a tester. Some days I'm a part-time ops person. Some days I'm a build manager. Some days I'm a release manager. Some days I've been a Scrum Master (and, in fact, I was a dedicated Scrum Master for three months). I was still a tester but I did what was needed for the moment and often that meant not being a "Tester" but always being a "tester"... see what I did there ;)?

Are test cases necessary? It depends on what you define as a test case. In my world, I go through a few waves of test case development. Almost never do I start with some super detailed test case. Typically I start with a 5000-foot view and then I look to get an idea of what is in that space. I may or may not even have a clear idea of how to do what I need to do, but I will figure it out. It's the process of that learning that helps me flesh out the ideas needed to test. Do I need to automate steps? Sure, but generally speaking, once I automate them, if I've done it right, that's the last time I need to really care about that level of granularity. Do I really care if I know exactly every step necessary to complete a workflow down to the property IDs needed to reference the elements? No, not really. Do I need to know that a unique ID name exists and can be used? Yes, I care a lot about that. In fact, that's about the most important finding we can make (see my talk about "Is this Testable?" about more of my feelings on this :) ).

The key takeaway, care more about the work and about being nimble than bowing to the altar of AGILE. I find much to value in that :).

Using Agile to Adapt to Changing Goals: an #STPCon Live Blog Entry

Using Agile to Adapt to Changing Goals

There's an old joke that goes around that says, basically, "If you don't like the way things are today, come back tomorrow". Meaning the only true constant is change and change happens a lot. We all have choices to be a driver of change or to have change be driven onto us. To this idea, any goals that your team has made are just as likely to change.


Sue Jagodzinski described her own situation of change within her company and the chaos that resulted from it. Part of her story described the challenges of managing two teams that were focused on different things. Both teams needed to use and support the adoption of a test automation tool and to be able to enhance and support that tool. One team focused on build, test, and automation. The other team focused on training, support, and documentation. While the tool they were working on was the same, they had very different missions and purposes and that would show up in the backlogs they built up and the priorities they placed on their respective backlogs.

Here's a bit of nerdy for you. The training and support team decided to call themselves Team1. In retaliation, the other team called themselves Team0 because they understood how arrays were indexed (and yes, this was a real dynamic between these teams. I thought she was kidding, she assured me she was not.

To this end, Sue determined that there were some key danger signs happening here. By segregating responsibility between two teams, there was an unhealthy competition that developed. trust issues developed along the way. when there issues there was plenty of finger pointing to go along with it. Most visibly was the fact that there were conflicts where teams would determine what to work on was guided by whatever the other team was not working on.

By moving to a shared backlog, the teams came into immediate conflict and had to negotiate how to change that dynamic. Some of the areas that Sue addressed was:

- how can we determine the skills that were needed on each team, and institute necessary moves if needed?
- determine the soft skill levels of the team members, who were the leaders? Who could become a new leader if needed?
- who would be best served to go elsewhere?
- how could we restructure the teams to be less contentious?

The last one was easy-ish to solve by changing names. the two teams for purposes of this presentation were "Awesomation" and "Thorium". Both teams agreed to go to a single backlog. Teams were set up so that both teams had technical expertise in the test framework. More to the point, an emphasis was made to reward and encourage those that would share their knowledge and skill sets. By doing this Sue was able to get close to equalizing the team. By sharing a backlog, the grooming sessions were done together, with their expected challenges. Sue took over as the product owner and she had to learn what that entailed. She described the process as being "harder than many people might realize" in addition to getting a better knowledge of the product.

The net results of this process, though not easy, were that they had the ability to learn how to do any task on either of the teams. In other words, two teams with Generalizing Specialists (service mark Alan Page ;) ). In the process, each of the team members engagement increased, different perspectives were heard, learning and reflection were done in the retrospectives, and the teams learned to progress together. Perhaps the most valuable skill they both discovered they could do was to adapt to priorities and be able to pivot and change if/when necessary.

What not to Test: an #STPCon Live Blog Entry

“Help, I’m Drowning in 2 Week Sprints, Please Tell Me What not to Test”

This talk title speaks to me at such a profound level. Be warned, I may veer into tangent territory here. There are what I cal "swells" that come with sprint madness and sprint fatigue. It's never constant, it's like a set of waves that you would look to time. For those familiar with surfing, this likely makes sense. For those not familiar, waves tend to group together and swells grow and shrink. These series of increasing and decreasing waves are referred to as "sets" and the goal is to time the set that feels good to you. Too early and you don't catch a wave. Too late and the wave wipes you out. In between is the rideable waves to drop in.

Sprints are the software mental metaphor that goes with "timing the waves" but the problem is that the timing of sprints is constant, but wave sets are not. Likewise, even figuring out what matters for a sprint may take more or less time any given sprint. Tasks like backlog grooming, story workshops, sprint planning, etc. all come down to making sure that we have an understanding of what matters and what's actually available to us.

Risk-based testing is the idea that we focus our attention on the areas that present the most potential danger and we work to mitigate that. We all know (or should know) that we can't get to everything. Thus we need to focus on the areas that really matter.

Mary recommends that we place emphasis on testing ideas. Testing ideas should go beyond the acceptance criteria. We can easily be swayed to think that focusing on the acceptance criteria is the best use of our time but often, we discover that with a little additional looking, we can find a variety of problems that simply looking at acceptance criteria won't cover. We also need to be aware that we can also range far afield, perhaps too far afield if we are not mindful. Test ideas are helpful but don't just play "what if" without asking the most basic question of "which would be the riskiest area if we weren't to address?"

An area that I find that happens (tangent time) is that I will be testing something and find that we have to deal with an issue related to our product ut has nothing to do with stories in play. As I am the owner of our CI/CD pipeline (note: that doesn't mean I'm the expert, just that I own it and I am the one responsible for it working properly). If something happens to our CI/CD pipeline, who do you think is the first person to spring into firefight mode? Are you guessing me? Congratulations! In a sprint, I don't have the luxury of saying "oh, sorry, I can't deal with pipeline issues, I have to finish testing these stories". Therefore, any time I have issues such as a pipeline problem that needs to be addressed, I immediately put a spike into the sprint. I do my best to consider how much time it will take and if I can handle it myself (often the case) or if I need to pull in development or ops resources (also often the case).  What happens over time is that we get a clearer picture of not just actual testing focus but also the legitimate interruptions that are real and necessary to deal with. In a sprint, there is a finite amount of time and attention any of us can spend. Time and attention spent on one area necessitate that it is not spent elsewhere and no, saying you'll stay up later to cover it is robbing your future self of effectiveness. If you are doing that, STOP IT!!!

Performing a test gap analysis is also helpful. In a perfect world, we have test cases, they've been defined and we have enough information to create automated tests around them as the functionality is coming together. Reality often proves to scuttle that ideal condition, or at least it means that we come up short a bunch. What we often discover is a range of technical debt. Areas may be well covered and easily documented with test cases and automated tests. Other areas may prove to be stubborn to this goal (it may be as simple as "this is an area where we need to spend some time to determine overall testability).

The Pareto Principle is a rule of thumb, it's not absolute. Still, the old adage that twenty percent of something is going to give you eighty percent of outcomes is remarkably resilient. That's why it's a rule of thumb in the first place.

Twenty percent of test ideas can help you find eight percent of the issues.
Twenty percent of the application features will be used by eighty percent of the customers.

What does this mean? It means you need to get a read on what's actually being used. Analytics and an understanding of them are essential. More important, using analytics on your test systems is important, not just the prod numbers. One thing that was driven home to me some time back was the fact that analytics need to be examined and the configurations need to be experimented with. Otherwise, yes, you can have analytics in place but do you actually know if you have them turned on in the right places? How would you know?

One more interesting avenue to consider is that you cannot test everything but you can come up with some interesting combinations. This is where the idea of all pairs or pairwise testing comes into play. Testers may be familiar with the all pairs terminology. It's basically an orthogonal array where you take a full matrix and from that matrix, you look at the unique pairs that can be created (some feature paired with some platform as an example). By looking for unique pairs, you can trim don a lot of the tests necessary. It's not perfect, and don't use it blindly. Some tests will require they be run for every supported platform and not doing so will be irresponsible. Still, prudent use of pairwise testing can be a huge help.

Seek JOY: #PNSQC Live Blog

Wow, we're at the end of the day already? How did that happen? Part of it was the fact that I started a few conversations with people that cut into talks being delivered, but as is often the case, those discussions can take priority and can often be the most important conversations you have at a conference.

Long story short though is that we are at the closing Keynote for the main two-day conference. Rich Sheridan of Menlo Innovations believes that we can do work that we care about and that we can have joy in the work that we do and in the workplaces we actively move in. Rich shared his story of how he came up into the world of computers and computing starting in the early seventies and how the profession he loved was starting to sap the life out of him and how he was contemplating leaving the industry entirely. He was experiencing the chaos of the industry. Issues, bugs, failed projects, blown deadlines, lack of sales, and all of the fun stuff any of us who have worked in tech recognize all too well. Chaos often ends up leading to bureaucracy, where we can't get anything done to not being able to get anything started.

The fact Rich wants to impart is that Joy is what all of us hope for in most of the things that we do. We often see it as some form but it's often nebulous to us. Additionally, jobs and companies cannot guarantee our success or our happiness. We have to have an active role in it and be willing to make it happen for us as we endeavor to make it work for others.

Why joy? Joy is service to others and being able to see the reaction to that service. IT's why we do the work that we do. We want to see our work out in the world. We want to see it get a response. We want to see people react to it. and we want to have that moment that swells up inside of us and that cheers us and makes us jump for (wait for it!) joy.

It's one thing to say that you want to build a joyful career, but it requires human energy. In most of the work environments that I have enjoyed the most, the work has almost always been secondary. What made the work enjoyable? The people and the interactions with those people are what makes for memorable experiences.

One of the most important things to foster joy is the idea of trust. We have to trust one another. Trust allows us to be open and frank. We can get into hard discussions and deal with conflict in a positive manner. When we can debate issues with trust and consideration, while still being committed to trying to get our issues resolved, we can deal with the hard issue and still be positive and remain friends.

Rich describes his office as a hodgepodge of machines, but the most astounding aspect is the fact that no one has their own computer. People change pairs and move onto other machines every five days, and with those moves, people move onto other machines. That means there is no such thing as "it works on my machine" because there is no dedicated machine for anybody.

Simplicity goes a long way to helping develop joy. Complexity for its own sake sucks the life out of us, and Rich showed the way that they manage their work. It's all on paper. It's all based on the size of commitment, and it's all based on creating meaningful conversations. Pans are then put on the wall, for all to see so that it's completely transparent. Honesty matters and with transparency comes honesty. With honesty and a willingness to express it, empathy comes into play. With empathy, we can see what others see and feel what others feel. To borrow from Deming, as Rich did "All anyone wants is to be able to work with pride" With that pride comes joy.

Rich Sheridan is the author of "Joy, Inc." and it is scheduled to be released very soon. Yep, I want a copy :).

And with that, the two-day technical program for PNSQC is over. Tomorrow will be another work day for me, but I think it's safe to say that I'll be literally buzzing with the kinetic energy that these past two days have provided for me. Thank you PNSQC team for putting on a great event. Thank you, speakers, for sharing your insights and experiences. Thank you, participants, for coming out and participating, and especially, thank you to everyone who came to my talk and offered positive feedback (and constructive criticism, too). Wishing everyone to get back to their destinations safely, and if you are here for the extended workshop day, I hope you all enjoy it and get to bring back great insights to your teams.

TESTHEAD out!!!

Owning Quality in Agile: #PNSQC Live Blog Day 2

Good morning everyone! I actually had a full night's sleep and no need to rush around this morning, so that was fantastic. I should also mention that I was really happy with the choice to go to Dar Salam restaurant last night as part of the Dine Around Town event. Great conversation and genuinely amazing food.

This morning, the theme for 2018 was announced. It will be "On The Road to Quality" and I am already thinking about what I'd like to pitch for a paper and talk next year. No guarantee it will be accepted, but I'm looking forward to the process anyway.

The first talk this morning is "Who Owns Quality in Agile?" by Katy Sherman and she started out with a number of questions for us. Actually, she started the morning with a dance session/audience participation event. No, I'm not kidding :).

Katy shared the example of her own company and the fact that her organization had two pulling interests. In some cases, testers felt they didn't have enough time to test. On the other hand, developers felt like they had too much time with nothing to do. One of the challenges is that there is still a holdover from older times as to what quality means. Is it conformance to requirements? Or is it something else? How about thinking of quality as meeting both the explicit and implied needs of the customers? We want conformance, we want user satisfaction and we want our applications to perform well, even if they don't know what performing well even means. Ultimately, what this means is that Quality cannot be achieved through testing (or at least not testing alone.

In some ways, there's still an us vs them attitude when it comes to being a developer or being a tester in an Agile team. For my view, silos are unhealthy and perpetuate problems. There's an attitude that testing in extra, that having a dedicated tester is a frustrating overhead and wouldn't it be great if we could do away with it? Some developers don't want to test. Some testers struggle with the idea of becoming programmers. Many of us are in between. As a tester that has reported to test managers and to development managers, there's a need to be direct and show what you can bring to the table and to express issues and concerns in a similar way (perhaps not exactly the same but feel confident that both can be taught to understand the issues). In my own work, I have determined that "being a tester" alone isn't a realistic option at my company. There is too much work and there are too few people to do it, so everyone needs to develop their skills and branch into avenues that may not necessarily be their specific job title. Consider the idea that we are all software engineers. Yes, I realize that engineer is a term that has baggage, but it works for the current discussion.

There is a lot of possible cross training and learning that can happen. Developers being taught how to test is as important as testers being taught how to code. Perhaps it is even more important. Testers can learn a lot about the infrastructure. Testers can learn how to deploy environments, how to perform builds, how to set up a repository for code checkout, and yes, the old perennial favorite, work with automating repetitive tasks. Please do not believe that more automation will eliminate active human testing. Yes, it is true that it may well end a lot of low-level testing and that it will eliminate a number of jobs focused on that very low level, but testers who are curious and engaged in learning and understanding the product are probably the least likely to be turned out. I'm mimicking Alan Page and Brent Jenson of A/B Testing at this point, but if testers are concerned about their future at their current company, perhaps a better question to ask would be "what value am I able to bring to my company and in what areas am I capable of bringing it?

One of the key areas that we can make improvements is in the data sets and the data that we use to both set up tests as well as to gather to analyze the output. Data drives everything, so being able to rive the testing by data and analyze the resulting data produced (yes, this is getting meta, sorry about that ;) ). By getting an understanding of the underlying data and learning how to manipulate it (whether directly or through programmatic means) we can help shape the approaches to test as well as the predictions of application behavior.

Is QA at a crossroads? Perhaps yes. There is a very real chance that the job structure of QA will change in the coming years Let's be clear, testing itself isn't going anywhere. It will still happen. For things that really matter, there will still be testing and probably lots of it. Who will be doing it is another story. It is possible that programmers will do a majority of the testing at some point. It's possible that automation will become so sophisticated and all-encompassing that machines will do all of the testing efforts. I'm highly skeptical of that latter projections, but I'm much less skeptical of the former. IT's very likely that there will be a real merging of Development and QA. In short, everyone is a developer. Everyone is a tester. Everyone is a Quality Engineer. Even in that worldview, we will not all be the same and do exactly the same things. Some people will do more coding, some people will do more testing, some people will do more operations. The key point will be less of a focus on the job title and a rigorously defined role. Work will be fluid and the needs of the point in time will determine what we are doing on any given day. Personally, that's a job I'd be excited to do.




This is a 90-minute talk, so I'll be adding to it and updating it as I go. More to come...

Gettin' Covey With It: #PNSQC Live Blog

This talk is called "7 Habits of Highly Effective Agile" which is a play on words of the Steven Covey book, hence my silly title. Yes, I'm getting a tad bit punchy right now, but I am lucky in that I get to have an energy boost. Nope, no food or drink or anything like that. I get to have Zeger Van Hese next to me doing his sketch notes.

I've waited years to actually see him do this. How am I going to concentrate on doing a live blog with that level of competition (LOL!)?

Seriously though, "Seven Habits" is a book from the 80s that is based on the idea of moving from dependence to interdependence with a midpoint of independence.  For those wondering about the original Covey Seven Habits, they are:

1. Be Proactive
2. Begin With the End in Mind
3. Put First Things First
4. Seek to Understand, Then be Understood
5. Think Win/Win
6. Synergize
7. Sharpen the Saw

The Agile Seven Habits break down to:

1. Focus on Efficiency and Effectiveness (make sure that your processes actually help you achieve these, such as cutting down, technical debt, providing necessary documentation, etc.)
2. Treat the User As Royalty (understand your customer's needs, and what you are providing for them)
3. Maintain an Improvement Frame of Mind (adapt, develop, experiment, learn, be open, repeat)
4. Be Agile, Then Do Agile (we have to have internalized and actively live the purpose before we can do the things to get the results. Communicate, collaborate, learn, initiate, respond)
5. Promote a Shared Understanding (develop a real team culture, break away from roles, stop thinking it's not my job, learn from each other, wear different hats and take on different tasks, pairing as a natural extension, determine what is important)
6. Think Long Term (develop a sustainable process, pacing that's doable and manageable, making habits that stick and persisting with them)

Well, there's my thoughts. and here's Zeger's. His are much cooler looking ;):

Keep Your Project off the Autopsy Slab: #PNSQC Live Blog

All right, my talk is over, and from what I can tell from the comments back it seems to have gone over well :). Lunch was good and we got to get into the 35th Anniversary cake. The committee also put out cupcakes and I'm pretty sure this had to have been put out there as a joke:

I am happy to hear that no Vegans were harmed in the making of these cupcakes ;).

The first talk for the afternoon is to talk about Pre-Mortems, with the subtitle of "Keep Your Project off the Autopsy Slab". To make it simple, there'are three basic things to consider in a pre-mortem:

1. Something has failed.
2. You and a group have determined and found the issue.
3. Steps are put in place so that the issue doesn't happen again.

Put simply, projects frequently fail, and they fail for predictable reasons. Sometimes they are event-based, such as a person quits, or the schedule changes, or an integration is needed. Sometimes it's people related, such as a skill set is missing, or someone is not dedicated to the project. At times the culture works negatively, such as fostering overt competitiveness or being overly skeptical. Additionally, the goal to get it out there overrides the willingness and ability to do the job right. Technology can be a problem, as in external third party software elements seem great on paper but we end up struggling to try to use it.

One additional issue that can sabotage a project is a lack of knowledge, specifically with the idea of "unknown unknowns" as in "we don't know it and we don't even know we don't know it".

If we are looking at a pre-mortem as a way to get a feel for where things are likely to go, we first have to identify as many risks as possible, but just as important is determining if, and how, it will be possible to mitigate those risks. Mitigation is an oddball term, in that it's not entirely clear as to what we are doing. Are we minimizing risk? Reducing risk? In general, it's lessening the impact of risk as much as possible.

There are eleven ideas to consider with pre-mortems:

1. What is the scope?
2. How do we determine the logistics?
3. What instructions do you send?
4. How can you imagine the project failing?
5. What are the reasons for potential failure (brainstorm)?
6. Now, let's create a master list (consolidate, de-duplicate and clarify).
7. What items can we remove from the list due to low impact (filtering)?
8. What reasons of potential failure could be real risks to the real project?
9. What does the group consider important? Reorder based on importance (VOTE).
10. Action Items to mitigate the top risks.
11. Action items need to actually be done. They are the whole point of the pre--mortem.

The Good, The Bad and the Ho-Hum: #PNSQC Live Blog Continues

One of my favorite parts of coming to PNSQC is the fact that it has a high rate of recidivism. A lot of the attendees have been several times and it's great fun getting back in touch and catching up between sessions. I met my friend Bill here at PNSQC in 2010, and what we find hilarious is that, even though we both live in the Bay Area, it seems we only see each other here at PNSQC ;).

Heather Wilcox leads off the breakout track talks with a talk called "Four Years of Scrum: The Good, The Bad and the Ho-Hum". For the record, I work in a Kanban shop currently but worked in a Scrum shop from 2011-2012. Prior to that, none of the companies I worked for could be considered Agile (not for lack of trying for the last company I worked for up through 2010). What I've experienced is that each company does Agile just a little bit differently, and in many cases, they think they are doing Agile, but not really.

Heather described how in the early part of the process of getting to Agile, they were often performing the rituals of Agile, but they were not really sharing those rituals with other teams. As I have experienced that in a small way in the past, it's not often a recipe for success, especially since you have to explain Agile principles in a non-Agile space. Sounds like it wouldn't be much of a big deal, but you would be surprised.

What Heather described was the way that they implemented and perfected their practices, and in some cases, they determined that they were going to do things that were non-standard and that they were OK with it. Her team does things that may be considered unorthodox. Their teams are somewhat Siloed, they use various Agile Methodologies, some teams do formal scrum or wing it on a theme of Scrum.

Her direct teams use standups, planning meetings, demos, retros, chartering and a focus on "The Goal" which is to respect and preserve Scrum ceremonies while being as mindful of time as possible. I like that Heather uses the terms "Scrum Ceremonies" so freely because, in many ways, we don't want to admit that we are somewhat beholden to "rituals". If they do a demo, they only demo when they have something interesting to show. Retros are done with the same rules and techniques each time so as to let the team know what they need to do and be prepared for. As long as it works, that will be what they do. If it stops working, they will change the process, so long as it gets them closer to the overall goal.

Does Scrum Always Work?

To put it simply, no. When predictable but not precise is important, or there are small changes being made that can be accomplished in a sprint or two, then it works very well (and that tends to correlate with my own experience). When dealing with a hard stop date or having to focus on a large project, and there just doesn't seem to be a way to break this down, Scrum itself starts to break down. Regardless, it's been determined that points matter, and that those points are vital to being consistent. Their planning improves considerably when points are adhered to. They also help avoid massive stories. In short, Scrum is a technique, it's a planning approach, it's a methodology, but it is not gospel and it need not be ironclad. There is a discipline requirement and it can be uncomfortable, but it allows for flexibility if done correctly.


-----

Just to let you all know it will be awhile until my next update because I'm delivering my talk next :). I will do a "somewhat summary" of that after the conference.

This Week on #TheTestingShow: Real Work vs Bureaucratic Silliness

I have come to realize that I am terrible in the shameless self promotion department. I have been actively involved the past several months with producing and packaging "The Testing Show", but I've not been very good about talking about it here. Usually, I feel like I'm repeating myself, or I'm rehashing what I already talked about on the episode in question.

When Matt takes the show on the road, however, I do not have that same issue. This most recent episode of The Testing Show, titled "Real Work vs. Bureaucratic Silliness" was recorded at Agile 2016 in Atlanta. As such, Matt was the only regular panelist there, but he put together a group consisting of Emma Armstrong, Dan Ashby, Claire Moss and Tim Ottinger. For starters, this episode has the best title of any podcast we've yet produced for The Testing Show (wish I came up with it, but it's Tim's title). I think it's safe to say all of us have our share of stuff we like to do, or at least the work that is challenging and fulfilling enough that we look forward to doing it. We also all have our share of "busywork" that is inflicted on us for reasons we can't quite comprehend. Tim puts these two forces on a continuum with, you guessed it, Real Work on one side, and Bureaucratic Silliness on the other.

I think there's an additional distinction that needs to be included in here, and that's when to recognize that some task, behavior, or check point actually is an occupational necessity, and when it is truly Bureaucratic Silliness (BS). In my everyday world, let's just say this tends to grade on a curve. Most of the time, BS comes about because of a legitimate issue at one point in time. Case in point; some years ago, in spite of an extensive automation suite and a robust Continuous Integration and active deploy policy, we noticed some odd things passing through and showing up as bugs. many of these things were not areas we had anticipated, and some of them just plain required a human being to look them over. Thus, we developed a release burn down, which is fairly open ended, has a few comments of areas we should consider looking at, but doesn't go into huge details as to what to specifically do. The burn down is non negotiable, we do it each release, but what we cover is flexible. We realized some time back that it didn't make sense to check certain areas over and over again, especially if no disruption had occurred around those features. If we have automated tests that already exercise those areas, then that likewise makes it a candidate to not have to be eyeballed every time. If a feature touches a lot of areas, or we have done a wholesale update of key libraries, then yes, the burn down becomes more extensive and eyeballing certain areas is more important, but not every release every single time.

Usually, when BS creeps into the system, it's because of fear. Fear of being caught in a compromising situation again. Fear of being yelled at. Fear of messing something up. We cover our butts. It's natural. Once or twice, or until the area settles down or we figure out the parameters of possible issues, makes sense, but when we forget to revisit what we are doing, and certain tasks just become enshrined, then we miss out on truly interesting areas, while we go through BS to the point that we would rather pull our intestines out with a fork rather than have to spend one more minute doing this particular task (yes, that's a Weird Al Yankovic reference ;) ).

Anyway, were I part of the panel, that's part of what I might have brought up for discussion? If you want to hear what the group actually talked about, well, stop reading here and listen to The Testing Show :).

Does the Agile Testing Process Work? - A #BAST Lean Beer Summary (1/5)

On Thursday, December 10, 2015, the Bay Area Software Testers Meetup Group got together and held its quarterly Lean Beer event. Lean Beer is the "evening equivalent" of Lean Coffee, where individuals come together and discuss topics of interest in a loose format. Topics are proposed, the participants vote on the topic that most interest them, and the topics are ranked based on the votes they receive.

Each topic gets a set time to be discussed and at the end of the time, a vote is made to see if the discussion should be continued. if majority votes to continue, more time is added to the clock and the discussion continues. If the votes indicate energy has been depleted for that topic, we close it off and move on to the next one.

We covered five topics at our most recent event, and based on the feedback, and as a point of reference for the members of the BAST Meetup that couldn't attend, these are summaries of what was discussed.

The top vote recipient, and therefore the first topic we discussed, was the "Agile Testing Process", or more to the point, "is there such a thing"? For several of the attendees, it seemed that the Agile Development process worked, but that testing was still treated as an after the fact activity. Isn't the point of Agile software development supposed to be that information about the product is discovered quickly, feedback received, and the ability to deliver software faster is primarily to make better quality software available?

Included in these questions is also the changes that many have witnessed at their respective companies as to not just how software development has changed, but how software testing at those companies has changed as well. For some companies, there is not a dedicated test team any longer, at least not one with a separate reporting structure. The idea of a separate test team and test manager has given way to having an integrated team of programmers and testers working together as much as possible, or to having test be an activity rather than a separate role.

Regardless of the team structure, testing is meant to happen, but how that testing is performed or who does it is open to debate. For those of us who work as testers in our given organizations, our efforts are often spread among various initiatives. Some of us focus most of our efforts on exploration, some of us are primarily working on automation. Some of us have the ability to get into the story writing process early, via the Three Amigos approach, and can apply our testing efforts early on during the requirements stage. Here, we are able to provoke requirements and provide thoughtful analysis and add value by helping see what aspects we need to consider up front before development even starts.

One of the comments that most of us agreed with is that having a dedicated (as in, involved) product owner made a considerable difference. Unit tests and Test Driven Development can be a big help towards designing a product that meets the requirements set forth by the stories. Having a system in place that allows for the software to be built and deployed in a timely manner can be a tremendous help to the process. Automation is important here, but more so to eliminating busywork and pushing things around than actually providing core testing. There is still a need for meaningful testing and applying testing skills to these stories, and part of that involvement is having the discipline to focus on just those stories that can be done in a particular sprint.

Concentration on Information Radiation

Ever have an experience at a conference that just sticks with you, that one thing that seems like such a little thing and then, as you consider it more and more, you just slap your forehead and say to yourself "for cryin' out loud, why didn't I think of that?!"

On Monday while I was at CAST, I was the room helper for Dhanasekar Subramaniam's tutorial about "Using Mind Maps for Mobile Testing". Much of the session was around heuristics for mobile testing and the ability to capture heuristics elegantly inside of mind maps. as part of the process, we spent a bit of time creating mind maps in XMind to use later with our chartered test sessions. I've done this before. I've even created a full mind map of the entire James Bach Heuristic Test Strategy Model (yes, one mind map and yes, when fully expended it is massive. Probably too massive). As we were working to create nodes and sub-nodes, Sekar pointed out that there were many labels that could be applied to the nodes, and that the labels were additive. In other words, each node could have several labels applied to them. 

As I was looking at this, and seeing labels such as pie chart fill, green check boxes, people silhouettes of several different colors, red x's, green yellow and red explanation points, and many others, I started thinking about how, by color and proximity, we could gauge how much coverage we have given a particular mindmap (or in this case, how completely we have applied a heuristic to testing) and what the results were for that. Instead of stopping to write down lots of notes, each node we were testing would get a label placed, and each label would have a semantic meaning. Green check box meant good, red X would mean failed or something wrong, a quarter pie chart would mean a quarter done, a yellow square would mean something that was a warning, but maybe not an error. Different color people icons would mean the person who performed that set of steps, and so on.

As I was looking at this, I joked with Sekar that we could tell the entire testing story for a feature or the way we applied a heuristic to a story in one place with one relatively small map. We both chuckled at that, and went on to do other things.

The more I thought about this, though, the more I liked the idea. In a previous company, we set up a machine and had a couple of flat screen monitors attached. These flat screens were placed in the main area and left on, cycling the images that were shown, only in this case, they were graphs and pages of results that were relevant to us. In short, they were acting as information radiators for our team. At a glance, we could know if the build had failed, if deployment was successful or not, and where the issue would be if there was one. We could also use this technique for information radiation. Imagine a charter or set of charters. Each one had their own mind map. Each mind map could be cycled through presentation on the monitor(s). the benefit would be that, at a glance, the team would know how testing was going fort that area, and we could update it all very quickly. I kept experimenting with it, and the more I did, the more I became convinced this just might work.

To that end, I am holding a Weekend Testing session this coming Saturday, August 8, 2015 at 10:00 a.m. PDT. We will look at mind mapping in general and XMind in particular, and we will develop a small heuristic for a feature (within XMind itself) to test and to update. I really like this idea, but I want to see if it can be tinkered with, and if it might be a useful approach to others.

If you think this might be a fun way to spend a couple of hours, come join us on Skype. Contact "weekendtestersamericas" and add us as a contact. On Saturday, get on Skype about 20 minutes before the session and say you want to be added to the session. Prerequisite, if you want to follow along and actually do the exercise, would be to download the XMind app for your platform.

 Again, I apologize for the short notice, but I hope to see you Saturday.

Exploring at Cloud Foundry - Live at #CAST2015

Jessie Alford told me something that astounded me... he's at Pivotal, working on Cloud Foundry... hey wait a minute, that means he's in my neck of the woods now! Mind you, that has nothing whatsoever to do with his talk which is "Driving Adoption of Chartered Exploratory Testing In An Agile Organization", but it made me smile because I  have talked with a n umber of people over the last couple of years about what Cloud Foundry does and how they approach testing... and now I can see it for real :).

Pivotal believes in exploratory testing, even if they don't have a dedicated test team. They have dedicated explorers that work in shifts along with their other development responsibilities. they create a variety of charters to help describe areas that might be risky.

I am impressed that a programming team would spent the time to develop charters for exploratory testing, but Pivotal has surprised me many times over the years (at my previous job I was a daily user of Pivotal Tracker). Jessie shared his backlog with listed charters, and it's cool to see how targeted they are.

Also what's cool is that everyone is encouraged to write charters, and each team develops their own norms as to how they are written. Each of the questions they ask are able to point back to the charters for targeted testing. As Jessie explained, charters are a general purpose scaffold. When Dev teams started looking at charters, they used them for testing, but they also started using them for lots of other things, like how to interact with other components, and use it to make sure that they are working effectively.

Another key aspect they learned was that pairing is not enough to communicate skills across an organization, but charters can allow teams to codify those abilities. Additionally, it's not enough to just teach exploratory skills, but to be able to confirm that those skills are transferred. In Pivotal, the term "test" doesn't get used very often in Pivotal, but "Explore" is an every day word. When you say test in a TDD, CI, CD environment, test has become so diffuse that it's lost its meaning, but exploring is easy to communicate, so it gets used. Automated checks are of course used and are important, but the ability to look at exploring makes clear the ways it is used in Pivotal, which is pretty cool.

Quick message to Natalie Bennett, Elisabeth Hendrickson and Dave Liebreich... Jessie done good, y'all should be proud of him. Jessie, if you find yourself down in the  Pivotal Palo Alto office, let me know so we can get together for lunch or something :).

The Case for "Just a Little More" Documentation

The following comment was posted to Twitter yesterday by Aaron Hodder, and I felt compelled not just to retweet it, but to write a follow up about it.

The tweet:

"People that conflate being anti wasteful documentation with anti documentation frustrate me."

I think this is an important area for us to talk about. This is where the pendulum, I believe has swing too far in both directions during my career. I remember very well the 1990s, and having to use ISO-9001 standards for writing test plans. For those not familiar with this approach, the idea was that you had a functional specification, and that functional specification had an accompanying test plan. In most cases, that test plan mapped exactly to that functional specification. We often referred to this as "the sideways spec”. That was a joking term we used to basically say "we took the spec, and we added words like confirm or verify to each statement." If you think I'm kidding, I assure you I'm not. It wasn't exactly that way, but it was very close. I remember all too well writing a number of detailed test plans, trying to be as specific as possible, only to have it turned back to me with "it's not detailed enough." When I finally figured out what they really meant was "just copy the spec”, I dutifully followed. It made my employers happy, but it did not result in better testing. In fact, I think it's safe to say it resulted in worse testing, because we rarely followed what we had written. Qe did what we felt we had to do with the time that we had, and the document existed to cover our butts.

Fast forward now a couple of decades, and we are in the midst of the "Agile age”. In this Agile world, we believe in not providing lots of "needless documentation”. In many environments, this translates to "no documentation" or "no spec” outside of what appears on the Kanban board or Scrum tracker. A lot is left to the imagination. As a tester, this can be a good thing. It allows me the ability to open up different aspects, and go in directions that are not specifically scripted. That's the good part.

The bad part is, because there's sparse documentation, we don't necessarily explore all the potential avenues because we just don’t know what they are. Often, I create my own checklists and add ideas of where I looked, and in the past I’ve received replies like “You're putting too much in here, you don't need to do that, this is overkill." I think it's important for us to differentiate between not overthinking and over planning, and making sure that we are giving enough information and providing enough details to be successful. It's never going to be perfect. The idea is that we communicate, we talk, we don't just throw documents at each other.  We have to make sure that we have communicated enough, and that we really do understand what needs to happen.

I'm a fan of the “Three Amigos” model. Early in the story, preferably at the very beginning, three people come together; the product owner, the programmer and the tester. That is where these details can be discussed and hammered out. I do not expect a full spec or implementation at this point, but it is important that everybody that comes to this meeting share their concerns, considerations and questions. There's a good chance we still might miss a number of things, because we didn't take the time here to talk out what could happen. Is it possible to overdo it? Perhaps, if we're getting bogged down in minutia and details, but I don't think it is a bad thing to press for real questions such as “Where is this product going to be used? What are the permutations that we need to consider? What subsystems might this also affect?" If we don't have the answer right then and there, we still have the ability to say “Oh, yeah that's important. Let's make sure that we document that.”

There's no question, I prefer this more nimble and agile method of developing software to yesteryear. I would really rather not go back to what I had to do in the 90s. However, even in our trying to be lean, and in our quest for a minimal viable products, let’s be sure we are also communicating effectively about what our products need to be doing. My guess is, the overall quality of what comes out the other end will be much better for the effort.