From Mid-Town Manhattan, it's #TestRetreatNYC

In the offices of LiquidNet, a group of intrepid, ambitious testers met and decided to discuss what aspects of testing mattered to us. Test Retreat is an Open Space conference, where the sessions begin when the participants want to have it begin (and end), the people who are there are the people who need to be there, and the law of two feet rules. We took some time to discuss topics, pull together similar topics and threads, and then head into our places to talk about the stuff that is burning us up inside.

---

The first session I attended was based around developing a testing community within a city or region. As many of us were part of different communities around the world, there were a variety of experiences, ranging from very small markets with perhaps a hundred testers total, to large regions with hundreds of thousands of testers, but little in the way of community engagement. 

Rich Robinson led the discussion and shared his own experiences with growing and scaling the community in Sydney, Australia. Rich shared that there were three areas that were consistently asked for and looked at as the goals of the attendees: looking for work, finding out the latest trends, and honing and sharpening skills. Rather than try to be all things to all people, they developed a committee and separate initiatives, with committee members focusing on the initiatives that mattered to them. For the group that wanted to get work, they made an initiative called “opportunity seekers”, and focused energy on those who had that as their biggest objective. For those who want to focus on latest trends, they have an avenue for that as well. 

Different regions have unique challenges. In the Bay Area, we have an overabundance of meetups for technical topics, of which a handful of them relate to software testing. As a founder of Bay Area Software Testers (BAST), Curtis Stuehrenberger, Josh Meier and I have chosen to try to focus on areas that are not as often discussed (we tend to steer clear of automation tools and techniques, since there are dozens of other meetup groups that cover those topics). Another challenge is frequency. In some cases, regular and frequent meetings are key. In others, having them less frequently works, but the key is that they meet regularly (monthly, every six weeks, or quarterly seem to be the most common models).

Rich also shared that their initiatives branch away from the formal meetup sessions, and have other opportunities that they initiate that occur outside of the formal meetup times. By having each initiative have people committed to it and resources to help drive those initiatives, buzz gets generated and more people get involved. One of the key things that Rich emphasized was getting people involved and engaged for these initiatives. The Sydney tester group has a committee of ten members that helps make sure that these initiatives are staffed and supported.

Another challenge is the local regions. Some cities have sprawl, others are difficult to get to in a timely manner due to traffic and population density. For example, the San Francisco Bay Area has four general regions: San Francisco (and the upper San Francisco Peninsula, Silicon Valley (and the lower San Francisco Peninsula), the East Bay, and the North Bay. With a few exceptions, people who participate in one generally do not regularly participate in the others. To reach out to a broader community in these sub-regions, it may require using technology and remote-access options for people to participate. 

Ultimately, growing a community takes time, it takes dedicated people, it takes a range of topics that matter to the attendees (including making sure that food and drink is there ;) ). To get those people you want to be involved, it helps to be very specific about what is needed. Saying “I need help with this” is less effective than saying “I need this specific thing to be done at this time for this purpose”. Specificity helps a lot when recruiting helpers.

—

The next session was “Sleep No More”, presented by Claire Moss, and focused on the model of the performance/play called Sleep No More (which is, in some ways, described as “an immersive performance of Macbeth”). It’s a darkened environment, all participants wear masks, no photography, no talking, just experience as the person sees it. Exploratory Testing, in many ways, has similarities to this particular experience. Claire used a number of cards to help display the ideas, and one of the first ideas she shared was “fortune favors the bold”. Curiosity and a willingness to go in without fear and deal with a substantial amount of “vague” is a huge plus. If you already have that, you have a strong advantage. If this is not natural for you, it can be developed.

Each room in the Sleep No More experience was part of the performance, and at any time, rooms could be empty or have people filter in during the performance. There are “minders” in the event that help to make sure that people don’t completely lose track of where they are. At times, there are very personal experiences that take place based on your tracks and where you go. Claire described a very intense experience of the performance based on where she went and what she observed and chose to follow up on. She also said that, up to this point, no one that she knew had anything like the experience she had.

The experience of Sleep No More was bizarre, creepy, full of strange triggers, and the potential to go into wildly unexpected direction. Software testing in many ways mirrors this experience. While there may be familiar areas and ideas, very often, our choices and angles may take us into very unexpected places. To give an example of the scope of the space, this was in a six story building, in an area that used to be a hotel (and whole areas of the building were gutted, in some spaces multiple floors were open to the air and visible). 

Claire described a feeling of “amazement fatigue”, where the level of stimulus is so high that there is no way to take it all in. The participants have to make conscious choices as to where they will go, and many of the participants will have wildly different experiences. Sometimes, they would follow a character, only to watch them go through a door and close and lock it, so that they couldn’t be followed any longer. This reminds me of following threads of a feature, and being brought to a dead end. People will observe different things, and they will also observe what other people do, and what they focus on. This can give us clues as to areas we want to explore next. 

This experience sounds amazing, and I am definitely interested in going and doing it myself, if time and commitments permit me to do so. Looks like I will be attending the August 10 performance :).

—

The next session was based on “Leadership”, and Natalie Bennett led the session with the idea that she wanted to see where individuals felt their experiences or needs for leadership were, as opposed to her telling us what she felt about Leadership and how to do it. Questions that Natalie wanted to discuss were:

- What is the purpose of a test team lead?
- What is it for?
What makes it different than being a test manager?

The discussion shifted from there into ways that test team leads and test managers were similar and where they differed. Some of the participants talked about how they led by example, and that they divvied up the work among the group based on the people involved and what they were expected to do. Team leads in general do not have hiring/firing authority, and they typically do not write reviews or have salary decision input. In other environments, the team manager and team lead are one in the same. There are some who are cynical about the effectiveness of this arrangement, while some feel that it is possible to be both a team lead and a team manager.   One attendee who is a Director of Q.A. for her company said that she was “the face of Q.A.” to the organization, and as such, she was setting the direction and expectations for the organization, as well as for her own direct reports.

Team leads are expected to teach and coach the members of their group, as well as be the point of contact for the group. It’s seen as important that they be able to focus on and develop their own role and make it responsive to their own environment. The team lead stands up for the group, and defends them from encroachment of issues and initiatives that are counter-productive to their success. Responsibility and authority tends to be on a sliding scale. Different companies allow for a different level of authority for the leads. Some give a level of authority that is just short of being an actual manager. In others, the leads is considered a “first contact” among equals.

One of the bigger challenges is to deal effectively when team members are failing. Failing in and of itself is not bad. It’s important to learn, and failing is how you learn, but when the failing is chronic or insurmountable, there needs to be a different level of interaction. Lean Coffee, direct mentoring, or even a serious re-consideration of experiences and goals can be hugely beneficial, both for the individual and the team as a whole.

—

Matt Heusser led a session about “Teaching Testing”, and some of the challenges that we face when we teach software testing to others. When we have an engaged and focused person, this usually isn’t a problem. When the person in question isn’t engaged, or is just going through the motions, then it’s a little more difficult.  The question we focused on at first was “what methods of teaching have worked for you?” Testing is a tactile experience, rather than looking at an abstract questions. We are familiar with questions like “how do you test a stapler?” or “how can you test a Rubik’s Cube?”. The presentation of this challenge may be the most important aspect. For some, they might look at “how do you test a stapler?” as demeaning. They are professionals, what is this going to teach me?

In my experience, one of the things I found to be helpful is to actually spell out how challenging the exercise could be. Rather than ask “How do you test a stapler?”, I might instead say “Tell me the 120 ways that you can test a stapler to confirm it’s fit for use?” This sets a very different expectation. Instead of saying “oh, this is trivial”, by seeding a high number, they may want to try to see how they might be able to meet or exceed that number. They become engaged.

To borrow a bit from Seth Godin, there are two primary goals for everyone. It’s the important aspects that we need to learn, regardless of the discipline. The first is to focus on authentic problems. The second is to be able to lead. Domain knowledge is a huge factor in helping to identify authentic problems. It’s not the only means, but getting to really know the domain can help inform the testing ability. Another important aspect is to understand how people learn, Everyone goes about learning a bit differently. Helping each person learn how they learn can be a huge step in helping to teach them. Sometimes the most ripe area of learning is to wade into an area where people disagree, or where there might be a number of people or groups where there might be dysfunction, where team members don’t talk to each other, or there’s simmering hostility between people. If there’s hostility between two programmers, and they write software that interacts with each other, it’s a good bet that there might be a goldmine of issues between their interaction points (I think this is a very interesting idea, btw :) ) .

Key to teaching testing is the ability to reflect and confirm what has been taught and learned, and for me, I think that Weekend Testing does this very well. The benefit of Weekend Testing, beyond just doing the exercise, is that we can see lightbulbs turning on, and there’s a record of it that others can see and learn from. Creating HowTo’s can also be a helpful mechanism for this. 

—

This section is the talk that Smita Mishra and I gave about “Hiring Testers and Keeping them Engaged Once We’ve Hired Them”. I recorded this session, and I will transcribe it later ;).

—

Claire Moss led a session on “Communicating to Management” and we went through and considered a list of questions that are important to frame the conversation(s):

What does quality like to our organization?

Why spend money on testing?

What does testing do?

What value are we getting out of testing?

I read this about QA, and it says we should do this… why aren’t we doing this?” 

These are all questions that we need to be prepared to answer. The question is, how do we do that?

There are several methods we can use, but first and foremost, we need to determine what we need to speak with management about, and if possible, use the opportunities to help educate them about what it is we can do, and at the same time, get a clear understanding about what their view of the world is.

Looking to standards and practices that are helpful can give us guidance, but it doesn’t always represent our reality. Information needs to be specific to explaining where we stand at the given time. Testing is primarily focused on giving quality information to the executives so that they can make qualified decisions. That is first and foremost our mission. Information that we can effectively provide is: 

- Framing of the ecosystem on a global scale (browser standards, trends, data usage histories)

- Impact on customers (client feedback, analytics data)

- Clarify issues and questions (heading off the executive freakout)

- Managing expectations (especially when dealing with something new)

- Explaining how likely issues brought to their attention really are problems worth investing in

- Explaining risk factors and methods to mitigate those risks

—-

At the end of the day we had a lot of new idea, feedback for some new initiatives, an emphasis on better communication, more focused due diligence, and the fact that so many participants had a lot they felt they could contribute. This was a fun and active day, and a lot of learning and connecting. One of the key things I am always impressed about when it comes to these events is that we really have a lot of solid people in the testing community, but we need even more.

I encourage every tester that admires craftsmanship, skill, and thinking make it a point to come to these now annual events (this is the third of these, so I think it’s safe to say that it’s a thing now ;) ). Once again, thanks Matt (Heusser) and Matt (Barcomb) for organizing what has becoming my favorite Open Space event. May there be many more.

Take the Risk of Thinking For Yourself: 99 Ways Workshop #40

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.

My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions.

Suggestion #40: Resist the temptation to go after bogus certifications. Take the risk of thinking for yourself. I've got 99 ideas on how to become a better tester, but certification ain't one. - Johan Jonasson

There is definitely a theme in the testing world, and no matter how much I want to shy away from it, I can't. Regardless of my personal connections to AST and teaching BBST courses, regardless of my own personal misgivings with the methodology and financial incentives associated with ISTQB and its certification model, the fact remains that there is a movement in the corporate world towards requiring certification/credentialing of people and skills. Much as I would love to ignore the issue entirely, the marketplace and the fact that certain companies in other parts of the world will not hire software testers if they do not have a certification means I have to speak to it.

This next suggestion is a big one. It's not going to be a simple "just go and do this, and you'll be a better tester". This is a lot bigger, and it's more than some techniques or workshop. This is a total paradigm change, a complete rewiring of your expectations and way of thinking. Even if you don't follow through on this one, at least consider it, and the ramifications of what you might to do make it happen.

Workshop #40: Have a body of knowledge/skill to show to others that would help you work without a resume, a degree, or a credentialing of any kind

This comes straight from Seth Godin's book "Linchpin". He deserves all the credit for this.

Imagine if, tomorrow morning, I were to go out and completely erase every online resume you've ever created. Removed every reference to any university degree, deleted any reference to any certification exam you've ever taken. In short, there's no paper trail or "official documentation" anywhere to justify who you are, what you do, or your value in your line of work.

How would that affect you?

In some jobs, it would be devastating. Lawyers cannot be licensed if there isn't proof they passed the bar exam. Doctors cannot practice medicine and dentists cannot work on teeth if they don't have a license. At this current point in time, there is no "credentialing" that is required by law anywhere for software development or software testing (at least none that I am aware of). If all reference to traditional means of "proving your worth" in the HR space were to disappear tomorrow, those of us who work with software testing would still be able to do our job, still be effective, still be very useful and of good service to those who would employ us.

Ah, but now you don't have a degree to reference, a credential to point to. Heck, you don't even have a resume to give anyone. If you stay where you are, that's probably not a big deal. If, however, you were interested in exploring a new opportunity... what would you do?

I said that I'd erased anything related to a credential or a resume. That's it. I haven't erased any of the actual work that you've done, or anything else related to it. In this brave new world, how would you get noticed?

Godin offers some great suggestions, such as:

- have five letters of introduction from the greatest thought leaders in your industry.

- have a live web site that has your product or some actual demo that anyone can go to and interact with in as much details as they might want.

- have a blog that is so compelling, people cannot help but read and take notice.

- participate in projects and do so well with them that your name becomes an industry standard.

These are all amazing ideas, and for most of us, we'd just shrug and say "well, yeah, all of that would be great... but I don't have any of those things".

Godin says to this "yeah, that's my point!"

This was the kick in the teeth that convinced me to start up TESTHEAD, and for me personally, many of the opportunities that I've been able to become a part of stemmed from this concept, and from that action.

Put yourself out there for the world to see. Give a lot of what you know, what you learn, and what you hope to learn out there (and do it for free/make it free). Get involved with a variety of initiatives, especially those that really excite you. Having a visible and actively growing body of work. Over time, this will help grow a reputation that makes people come to you. It's a virtuous cycle; the more we put out there, the more opportunity seems to come our way (well, so far, that's how things have worked for me, and yes, I want to see it keep on working that way ;) ).

Bottom Line:

The whole point to this thought experiment and potential outcomes (github repository, blog that gets many interested and influential readers, product demo, thought leader recommendations, record of written articles, videos of presentations, slide decks, etc.) is to change the dynamic of the discussion entirely.

Not everyone is going to want to do this. It requires a lot of commitment, a desire to be involved in the broader community, and a fair bit of personal marketing to get your name and "brand" out there. Doing so, though, will show that a "self credentialed tester" can stand toe to toe with anyone else out there.

STAREAST Day 2 Recap

I have to admit, it's a challenge to separate the fact that I am on East Coast time from the realities of waking up at 7:00 AM each morning, which is actually 4:00 AM where I'm from. I do that from time to time, but not with such late evenings. Still it's been a great deal of fun to hang out with such a great group of people and to have such wonderful, serendipitous moments.

The day started out with breakfast and a keynote address from Dorothy Graham of Grove Consulting about "What Managers Think They Know about Test Automation–But Don't". Dorothy points out that often, Management is looking for the following aspects to happen when they talk about test automation. They believe:

• automation can test everything
• automation will find all the bugs
• with automation we will get test coverage of 100%
• more automation means more confidence
• run tests overnight and weekends
• automation will prevent human error

The fact is, there's a lot more that goes on in these areas than what management hopes to have happen. Yes, automation can save time, but it will only save time after the automation is developed, and it won't save time if the development of the automation is close to the amount of time it takes to test things manually. Trust me on this, it takes time to automate tests well. The architecture of well-running automated tests is key. making that architecture takes time. Poor architecture often leads to poor automated testing, and frequently to abandoned automated testing. The relationship with the development team is also critical. If we work with them, we can develop solid tests that can be useful for the long term. If we develop our tests in isolation, we may miss a lot of things, but more importantly, we may find we are working at cross purposes and not realize it.


James Bach gave an excellent talk on "The Dirty Secret of Formal Testing". What made this talk interesting was the fact that he was describing testing in highly regulated markets, dealing with a product that was very life or death (a medical device to help aid patients with heart problems). These environments are extremely formal, but the dirty secret of these environments is that informal testing goes on all the time. It has to. Otherwise there's no way to learn about what the product actually does. As described by James, formal testing is any testing that must be done a specific way or must check specific facts. Requirements as described by James (and I like this description) are ideas at the intersection between what we want, what we can have, and how we MIGHT decide that we got it. In complex and life-or-death systems, we have to test to find out if that has happened, and we have to test first to find out if we actually accomplished those goals. Thus, contrary to popular belief and conventional wisdom, good formal testing MUST first begin with informal testing.

In addition to the variety of sessions, I have to point out that many times, the best discussions and best ideas happen between the sessions, or happen when we decide to engage in conversations with a single person or a handful of people, effectively creating our own session on the spot. I confess, I had a hidden motive with coming to STAREAST beyond giving my talk and learning some cool new ideas to bring home. My goal was to see what I could learn from many different people about how to teach software testing, and make it fun, for 16-24 year olds. More to the point, I needed to recruit helpers willing to do the work, and I was not disappointed. Instead of me looking for people to discuss this idea, word preceded me, so I had people coming up to me to ask if they could take part. Based on the interactions I had with many people this week, I got what I came for and plenty more. I had some terrific discussions with people from many industries as well as educators who have helped me narrow down and distill some of the ideas I have, and it's given me many new directions to consider. They may not all be possible, but I'm looking forward to trying them out.

STAREAST Virtual was a fun and interesting way to communicate with the audience of participants that were not specifically at the conference, and it was fun to have Matt Barcomb participate with me in a conversation about Testing Roles and Approaches to Paired Development and Testing. In addition, we also discussed many of the aspects of collaboration and ways to break down the unintentional walls that we build. In this way, we can set up our environments and approaches so that we can become more effective.

The main conference came to a close with the awarding of a number of awards, including those who found interesting bugs in the Test Lab, and something that made me smile greatly. It was announced to everyone just before the closing keynote that my paper and presentation were chosen as Best Paper for the conference :). For someone who gave his first official full hour-long track talk at any conference, well, yesterday, that was a wonderful thing to see happen (I rank it up there highly with "firsts" of my career :) ). I appreciate the award and those who felt I deserved it. Thank you very much!

The final keynote of the conference was given by Theresa Lanowitz of voke, inc., and it discussed "Testing Trends: Cloud, Virtualization, and Mobility". It gave a forecast of what we might see in the next two to three years, and I must say, things look exciting for testers, regardless of what the "test is dead" people are saying :). Numerous examples of performance problems, unintended consequences, security breaches and other issues of integration were explored, and they all pointed to the same thing. Testing needs to be performed, and it needs to be performed at a level that people can make solid and intelligent decisions about the ramifications and consequences. The simple fact is that there are no end of authentic problems to be tackled and the scope and the landscape are expanding. There will come a time when we will need more testers, not less. Get ready, folks :).

My thanks to so many people for the time they took to get to know me, speak with me, hang out in the hotel lounge or go out to dinner with me and talk about ideas and concepts that we all deal with and the solutions we hope to achieve. My thanks to Scott Barber, Claire Moss, David Gilbert, Matt Barcomb, Lee Copeland, James and Jon Bach, Lanette Creamer, Rachelle Sawal, Janet Gregory, Allison Wade, Matt Barcomb,  Randy Rice, Mirkaya Capellan, James Lyndsay, Michael Bolton, Zeger Van Hese, Griffin Jones, all the attendees of my talk about Weekend Testing, and everyone else that helped make this a memorable week.

'Til we meet again :).

STAR East: Here I Come!!!

My apologies for being so quiet the past week, but since I am going to be spending the next four days either in the air or at STAR East in Orlando, Florida, I had to focus on some other things. I did get to discover something really cool. Early last week, I received a check in the mail. When I saw that it was from Software Quality Engineering, I was curious to see what it was. The note for the check said "Best Paper", which I will interpret to mean that I received an award for the talk and paper that I presented as a Best Paper. I don't know if it is a Best Paper or the Best Paper, but either way, I'm excited to be picked for such a thing; it's another first for me :).

For those who will be at STAR East this week, I will be giving my presentation about "Delivering Quality One Weekend At a Time". If this title sounds familiar, you would be right. It was the paper that I originally wrote for the Pacific Northwest Software Quality Conference in 2011, but had to bow out of attending because of my broken leg and inability to walk well at the time. A co-presenter at PNSQC felt bad that I couldn't present my paper, and they contacted the folks at SQE and recommended they consider my paper as a talk. From there, I got the chance to have it submitted again, accepted, and now it's been voted a Best Paper. This tells me that I really have to deliver this talk, and now we are down to just a little over 48 hours until I get to deliver it :).

I'll be posting some details from the conference but I won't be able to do the extended "live blog" format I'm used to (they are not providing WiFi for the conference, and I'm one of those cheapo's who hasn't jumped on having an iPhone or Android yet ;) ), so my status reports will likely be posted nightly. As for now, it's one more mad dash day to get as much done on the home front before I leave. Hello, I must be going (LOL!).

Pair, Learn, Present?

Weekend Testing helped usher in a way for software testers to learn and practice their craft and build their technical skills. Now, a new initiative is underway and looks promising as the next step in bringing testing (and more testers) into the public eye.


Ajay Balamurugadas and Brian Osman are helping lead the next wave of software testing craftsmanship. This time, instead of addressing the workmanship and craft of testing (which Weekend Testing is well situated to help build and grow), Pair, Learn, Present is meant to help testers develop their skills as presenters (like, in public, with a full room of people, on testing and technical topics).

Are you interested yet?

Pair Learn Present (PLP) is being developed as a safe environment where testers can learn presentation skills by pairing with a partner, taking on a topic, and then presenting that topic to the world (through webinars or live presentations where appropriate). The idea and format is similar to Weekend Testing and espouses the idea of pairing testers to share knowledge and develop skills, however PLP will be focusing on developing presentation skills.

So if this sounds like an exciting idea to you (and it definitely sounds exciting to me :) ), then send an email message to PairLearnPresent [at] GMAIL [dot] com and tell them you want in. From there, they can help you with selecting topics and pairing with other testers.


You can also connect with PLP here:

On Twitter: @PLP_Learning
On Facebook: https://www.facebook.com/profile.php?id=100003306257247
On Skype: PairLearnPresent

Is 2012 the year you plan to break out of your shell and start presenting? If you've been hesitant or unsure, this may be a great way to get your feet wet. I've already told them my interest, so if you're looking for a partner to present with, hey, I'm available :).