STAREAST Day 2 Recap

I have to admit, it's a challenge to separate the fact that I am on East Coast time from the realities of waking up at 7:00 AM each morning, which is actually 4:00 AM where I'm from. I do that from time to time, but not with such late evenings. Still it's been a great deal of fun to hang out with such a great group of people and to have such wonderful, serendipitous moments.

The day started out with breakfast and a keynote address from Dorothy Graham of Grove Consulting about "What Managers Think They Know about Test Automation–But Don't". Dorothy points out that often, Management is looking for the following aspects to happen when they talk about test automation. They believe:

• automation can test everything
• automation will find all the bugs
• with automation we will get test coverage of 100%
• more automation means more confidence
• run tests overnight and weekends
• automation will prevent human error

The fact is, there's a lot more that goes on in these areas than what management hopes to have happen. Yes, automation can save time, but it will only save time after the automation is developed, and it won't save time if the development of the automation is close to the amount of time it takes to test things manually. Trust me on this, it takes time to automate tests well. The architecture of well-running automated tests is key. making that architecture takes time. Poor architecture often leads to poor automated testing, and frequently to abandoned automated testing. The relationship with the development team is also critical. If we work with them, we can develop solid tests that can be useful for the long term. If we develop our tests in isolation, we may miss a lot of things, but more importantly, we may find we are working at cross purposes and not realize it.


James Bach gave an excellent talk on "The Dirty Secret of Formal Testing". What made this talk interesting was the fact that he was describing testing in highly regulated markets, dealing with a product that was very life or death (a medical device to help aid patients with heart problems). These environments are extremely formal, but the dirty secret of these environments is that informal testing goes on all the time. It has to. Otherwise there's no way to learn about what the product actually does. As described by James, formal testing is any testing that must be done a specific way or must check specific facts. Requirements as described by James (and I like this description) are ideas at the intersection between what we want, what we can have, and how we MIGHT decide that we got it. In complex and life-or-death systems, we have to test to find out if that has happened, and we have to test first to find out if we actually accomplished those goals. Thus, contrary to popular belief and conventional wisdom, good formal testing MUST first begin with informal testing.

In addition to the variety of sessions, I have to point out that many times, the best discussions and best ideas happen between the sessions, or happen when we decide to engage in conversations with a single person or a handful of people, effectively creating our own session on the spot. I confess, I had a hidden motive with coming to STAREAST beyond giving my talk and learning some cool new ideas to bring home. My goal was to see what I could learn from many different people about how to teach software testing, and make it fun, for 16-24 year olds. More to the point, I needed to recruit helpers willing to do the work, and I was not disappointed. Instead of me looking for people to discuss this idea, word preceded me, so I had people coming up to me to ask if they could take part. Based on the interactions I had with many people this week, I got what I came for and plenty more. I had some terrific discussions with people from many industries as well as educators who have helped me narrow down and distill some of the ideas I have, and it's given me many new directions to consider. They may not all be possible, but I'm looking forward to trying them out.

STAREAST Virtual was a fun and interesting way to communicate with the audience of participants that were not specifically at the conference, and it was fun to have Matt Barcomb participate with me in a conversation about Testing Roles and Approaches to Paired Development and Testing. In addition, we also discussed many of the aspects of collaboration and ways to break down the unintentional walls that we build. In this way, we can set up our environments and approaches so that we can become more effective.

The main conference came to a close with the awarding of a number of awards, including those who found interesting bugs in the Test Lab, and something that made me smile greatly. It was announced to everyone just before the closing keynote that my paper and presentation were chosen as Best Paper for the conference :). For someone who gave his first official full hour-long track talk at any conference, well, yesterday, that was a wonderful thing to see happen (I rank it up there highly with "firsts" of my career :) ). I appreciate the award and those who felt I deserved it. Thank you very much!

The final keynote of the conference was given by Theresa Lanowitz of voke, inc., and it discussed "Testing Trends: Cloud, Virtualization, and Mobility". It gave a forecast of what we might see in the next two to three years, and I must say, things look exciting for testers, regardless of what the "test is dead" people are saying :). Numerous examples of performance problems, unintended consequences, security breaches and other issues of integration were explored, and they all pointed to the same thing. Testing needs to be performed, and it needs to be performed at a level that people can make solid and intelligent decisions about the ramifications and consequences. The simple fact is that there are no end of authentic problems to be tackled and the scope and the landscape are expanding. There will come a time when we will need more testers, not less. Get ready, folks :).

My thanks to so many people for the time they took to get to know me, speak with me, hang out in the hotel lounge or go out to dinner with me and talk about ideas and concepts that we all deal with and the solutions we hope to achieve. My thanks to Scott Barber, Claire Moss, David Gilbert, Matt Barcomb, Lee Copeland, James and Jon Bach, Lanette Creamer, Rachelle Sawal, Janet Gregory, Allison Wade, Matt Barcomb,  Randy Rice, Mirkaya Capellan, James Lyndsay, Michael Bolton, Zeger Van Hese, Griffin Jones, all the attendees of my talk about Weekend Testing, and everyone else that helped make this a memorable week.

'Til we meet again :).

STAR East Day 1 Recap

Yesterday was the first official day of the STAREAST Conference (not counting training and tutorial days ). I would have had this posted earlier, but I spent an entirely way too late night with a bunch of madcap testers... eh, what can you do ;)? .

We started out the day with breakfast, getting registered, and me getting my name tag with a beautiful little attachment. It felt really good to see the tag that says "Speaker" on the bottom :).

First off, right out the gate, we heard from Keith Klain from Barclays and the details of how he has been developing the Global Test Center for Barclays Capital, and the areas that they decided to work on to grow their testers. I found this to be very encouraging, in that a large corporation was actually looking at getting rid of metrics, bug counts and numbers that would seem to be logical, but turn out to not be effective. The talk focused on "Leading Cultural Change in a Community of Testers", and this made the point that, to make such a change, there had to be a large scale effort happening at the top level of the organization. One of the quotes that lead off this discussion was from Dwight D. Eisenhower; "“Leadership is the art of getting someone else to do something you want done - because he wants to do it.”

They focused on developing honesty, integrity and accountability in their testers. In return, they opted to jettison needless Test Maturity Models, Metrics Programs, and arbitrary "Career Development" approaches that are based on numbers, but not on people. Some of the principles they espouse are:

• People start to ignore testing when it is no longer relevant (in short, make sure the work you are doing is relevant) 

• Being responsible sometimes means rocking the boat (you have to drive the change, and sometimes that change will upset people)

• No one has the market cornered on good ideas (you'd be surprised who can come up with an amazing insight or twenty ;) )

• Never stop asking why – question everything (making assumptions or accepting things too readily can lead to blind spots)

• Invest 80% of your energy in your top 20%... (daring, but it makes sense; the people who perform are going to be the ones open to learning and growing)

• Leadership = Simplification (actually, leadership often means clearing obstacles and then getting out of the way)

• Don’t take it personally (you may be right, you may be wrong, you may be overruled, just remember it's not about you, it's about the product, services and customers)

• Think first – then do! 

I liked the fact that Keith included the technique about Start - Stop - Continue and what they would do (it warmed my Wood Badge heart to see that ;) ):

Stop 

Thinking that the value of the test team is in anyone else's hands and pretending “maturity” driven test metrics will make improvements

Start 

Telling the team exactly what's expected of them, supported by systematic training of testing skills, test reporting and business alignment

Continue

Driving out fear of failure by creating an environment that enables innovation and rewards collaboration through strategic objectives and constant feedback

Overall, this was a nice rallying cry to the troops. It was nice to see a senior member of the team showing focus on testers and testing, and point out that it's not just that we test, but that we develop meaningful skills and add meaningful value to the team.

Michael Bolton followed up with the 2nd Keynote of the day, titled "Evaluating Testing: The Qualitative Way". This was an interesting talk comparing the physical sciences to the social sciences, and the fact that quantitative examination with metrics and numbers makes sense when we are dealing with things like machining or physics, or other physical properties. Software isn't a true physical science,  it's actually more of a social science, because we deal with the interactions and the feelings of people to inform our tests. Michael quoted Dr. Cem Kaner's talk in which he stated that often times we don't get complete and fully fleshed out answers, but that we often get "partial answers that might be useful". Thus, we need to use different tools to evaluate software testing, or at least we need to focus on different ways of performing our testing that will allow for this social sciences level approach.

The key idea behind Qualitative Analysis is that Qualitative approaches are based on observation, making distinctions of categorization and classification as well as description and narration. By contrast, Quantitative approaches tend to assume that categorizations are accurate, and largely ignore associations with the object of observation. The key takeaway from this talk was that we needed to add the human element to our testing, and to do that well, we need to step away from the numbers and the left brain approaches, and get in and focus on what real people actually want and do (note: they are not always what we think they are). Great stuff from Michael as always (I've heard him do keynotes twice, and so far, I think he's two for two :) ).

Janet Gregory did a track session on Agile Testing practices, and since I had a chance to interact with Janet at the POST peer conference in Calgary, Alberta back in March, I was curious to hear a longer version of her ideas and approaches related to Agile Testing. One of the key areas that Janet wanted to point out was that, unlike in traditional development organizations, where roles are separate and unique, in Agile teams, roles are inter-related and overlap. It's not uncommon to expect that a Programmer, Domain Expert and Tester would overlap, and in many cases, reside in the same person. It's not necessary that everyone be capable of having that level of ability to cover all of those areas, as long as the organization can cover it with that level of overlap.

What this does tell us is that we are reaching a point of convergence for may of the people on these teams, so that a Tester is not just a tester. Key takeaways from this session were on how to implement Acceptance Test Driven Development and Behavior Driven Development. Put your tests first and then code so you know what's being covered. In addition, testers are NOT responsible for quality (the whole team is). Programmers do not code alone (everyone helps them understand what to code), and that the team needs the “right” roles and people (meaning we as testers may need to up our game to take part in these areas. 

The next session was a talk about Weekend Testing and it was delivered by, well, me :). I have written extensively about this topic so I won't recap the entire talk, but I will point out that the goal of this session was to encourage people to learn more about Weekend Testing and, in addition, learn how to bring the approach we used back to their own test teams. By all accounts I think it went really well. One funny comment. Apparently, because of my energy levels, people thought I was a consultant selling my services. They missed the several times I mentioned that Weekend Testing was a free endeavor. They kept waiting for me to say how much I charged, and still didn't quite believe me when I finished the talk and didn't say anything about charging for my services. Maybe giving away the SideReel swag was to blame, I don't know.

I closed out the day by conducting an interview with Lanette Creamer for the STAREAST Virtual conference. I don't know if that interview was just "live" and sent out over the ether, or if it was recorded and will be available later. Once I know, I'll let you know.

The official day closed out with the ever popular "Lightning Strikes the Keynote" which is where 8-9 participants present their best ideas or most compelling content, one after another, and are only allowed 5 minutes each. It certainly kept the energy level high to do that, and made for an enjoyable series of talks. 

While the sessions are definitely valuable, I have to say that I have found the best interactions to come with just talking with the other participants at times outside of sessions. Hey, imagine that, going to a conference to Confer, what a concept :). I've been greatly enjoying my discussions with Matt Barcomb of Lean Dog Consulting. We've been able to discuss a lot of aspects of challenges that Lone testers face, and what we can do to better interact with our teams. Oh yes, there will be blog posts on this forthcoming :).


Testers and tester games go together like, well, pick you favorite comparison, but it's inevitable. I had the chance to work with James Bach, Griffin Jones and Michael Bolton on a few different challenges. Some were frustrating, each made me question how I approached things, and I was quite happy when I figured them out (and walked a couple of other testers through the exercises and saw their different approaches). In the end, I went to bed "way too late" but fully energized from the interactions. Now it's time to see what Day 2 has in store. See you soon!

STAR East: Here I Come!!!

My apologies for being so quiet the past week, but since I am going to be spending the next four days either in the air or at STAR East in Orlando, Florida, I had to focus on some other things. I did get to discover something really cool. Early last week, I received a check in the mail. When I saw that it was from Software Quality Engineering, I was curious to see what it was. The note for the check said "Best Paper", which I will interpret to mean that I received an award for the talk and paper that I presented as a Best Paper. I don't know if it is a Best Paper or the Best Paper, but either way, I'm excited to be picked for such a thing; it's another first for me :).

For those who will be at STAR East this week, I will be giving my presentation about "Delivering Quality One Weekend At a Time". If this title sounds familiar, you would be right. It was the paper that I originally wrote for the Pacific Northwest Software Quality Conference in 2011, but had to bow out of attending because of my broken leg and inability to walk well at the time. A co-presenter at PNSQC felt bad that I couldn't present my paper, and they contacted the folks at SQE and recommended they consider my paper as a talk. From there, I got the chance to have it submitted again, accepted, and now it's been voted a Best Paper. This tells me that I really have to deliver this talk, and now we are down to just a little over 48 hours until I get to deliver it :).

I'll be posting some details from the conference but I won't be able to do the extended "live blog" format I'm used to (they are not providing WiFi for the conference, and I'm one of those cheapo's who hasn't jumped on having an iPhone or Android yet ;) ), so my status reports will likely be posted nightly. As for now, it's one more mad dash day to get as much done on the home front before I leave. Hello, I must be going (LOL!).

TESTHEAD On The Road

It seemed like such a little thing at the time :)!

Note, I'm not complaining, I'm happy to have the opportunities that come with being a passionate advocate, but sometimes I wonder when and how my reality got to be this packed so close together.

First, I'm very excited to be doing my first facilitation of a peer conference. My friends Lynn McKee and Nancy Kelln invited me to come up and act as facilitator for The Calgary Perspectives on Software Testing Workshop (POST). It's an annual peer workshop for software test practitioners, most attendees are from Canada, but some others are traveling from elsewhere. I'm excited to have been asked to participate, and I'm looking forward to being of service.

I'm home for just a couple of days, and then I fly out again to New Orleans for the Board Meeting for the Association for Software Testing, and while I'm there, I'm hoping to see if there's something I can do that Monday so that I can get to know some more people in the Software Test Professionals' Community attending STPCon (it's a tutorial day, and I'm not officially signed up for the conference, but I am hoping I'll be able to get involved some how for a little bit and get to meet and talk with people there.

I get a to week respite, and then it's of to Orlando, Florida for STAREast. I'm excited for this because of two reasons. First, I'm honored and privileged to have been chosen to present, but additionally, it's my second chance to present my talk on Weekend Testing that I couldn't present due to my broken leg last year (I had originally written it for the Pacific Northwest Software Quality Conference in 2011 in Portland, Oregon). Lee Copeland liked the content of the talk and asked if I'd be willing to present it at STAREast, to which I said yes, and thus, I'll be there to do so.

Truly, that's more traveling in a five week period than I have ever done before. It's been a challenge getting the rest of my life to align with these realities, but it's also a thrill that I have these opportunities in the first place. I've often joked that one could find opportunities in many places if they were willing to lift up the rocks and look for them. In many cases, when people know what you are up to and what you are willing to do and represent, they will also seek you out and give you the chance to make good on participating in those opportunities. I feel a bit overwhelmed at the immediate moment, but it's a good overwhelmed. It feels like I'm needed, and that people want to hear what I have to say and what I can contribute to the greater cause of testing. For that, I am both excited and grateful.

Three Cheers for Second Chances

So as you all know, the Pacific Northwest Software Quality Conference for 2011 just ended yesterday. Many of you also know I was scheduled to be there as a speaker. I had to drop out of the conference due to the condition of my leg, or the anticipated condition of my leg. As I look at it right now, I probably could have muddled through, but it would have impacted my delivery style seriously, so I still think I made the right decision. Besides, it's not the bones that are the issue, it's the circulation. If I walk, I'm OK. If I stand still, then I'm in trouble; circulation is not working well at the moment, and it starts to hurt after awhile. Anyway, what's done is done, and good or bad, I have to stand behind the decision.

Well, there were some people who were bummed about my not being able to give my talk. A good friend who shall remain nameless (unless they want to out themselves :) ), contacted me and asked me if I knew Lee Copeland, who chairs the STAR conferences (STAR-East and STAR-West). When I told them that I didn't, they said they'd get in touch with Lee and tell them about my situation and send them a copy of the original talk I had prepared. Which they did.

I'm happy to say that Lee got back to me and said that the committee for STAR-East reviewed my paper and proposal and thought it would be a good fit for the Star-East Conference in April. Of course, having just come off of a month's working from home, I felt strange asking for the time to go to the conference, but a commit was needed from me by a certain date, so I put it in and figured, well, we'll see what happens. After some back and forth, I got the clearance to go, and if I promote SideReel while I'm there, I'm on the clock. I think I can do that :).

Thus, I will get another chance to present the talk that I prepared for PNSQC in a live venue, and that live venue will be the STAR-East Testing Conference, to be held April 16-20, 2012 in Orlando, Florida. I've never been to a STAR-East event (OK, before 2010, I'd never been to a testing conference at all, so I think there may well be many "firsts" in this list ;) ). Nevertheless, there's a number of testers I've grown to know and appreciate over the years and several of them will be at STAR-East, so I'm looking forward to meeting and interacting with them, as well as getting the change to present the talk I've been itching to deliver now for several months. And huge thanks to a friend willing to help a guy out when he's down, too. So I guess my last question is... will I see YOU there :)?