Exploring at Cloud Foundry - Live at #CAST2015

Jessie Alford told me something that astounded me... he's at Pivotal, working on Cloud Foundry... hey wait a minute, that means he's in my neck of the woods now! Mind you, that has nothing whatsoever to do with his talk which is "Driving Adoption of Chartered Exploratory Testing In An Agile Organization", but it made me smile because I  have talked with a n umber of people over the last couple of years about what Cloud Foundry does and how they approach testing... and now I can see it for real :).

Pivotal believes in exploratory testing, even if they don't have a dedicated test team. They have dedicated explorers that work in shifts along with their other development responsibilities. they create a variety of charters to help describe areas that might be risky.

I am impressed that a programming team would spent the time to develop charters for exploratory testing, but Pivotal has surprised me many times over the years (at my previous job I was a daily user of Pivotal Tracker). Jessie shared his backlog with listed charters, and it's cool to see how targeted they are.

Also what's cool is that everyone is encouraged to write charters, and each team develops their own norms as to how they are written. Each of the questions they ask are able to point back to the charters for targeted testing. As Jessie explained, charters are a general purpose scaffold. When Dev teams started looking at charters, they used them for testing, but they also started using them for lots of other things, like how to interact with other components, and use it to make sure that they are working effectively.

Another key aspect they learned was that pairing is not enough to communicate skills across an organization, but charters can allow teams to codify those abilities. Additionally, it's not enough to just teach exploratory skills, but to be able to confirm that those skills are transferred. In Pivotal, the term "test" doesn't get used very often in Pivotal, but "Explore" is an every day word. When you say test in a TDD, CI, CD environment, test has become so diffuse that it's lost its meaning, but exploring is easy to communicate, so it gets used. Automated checks are of course used and are important, but the ability to look at exploring makes clear the ways it is used in Pivotal, which is pretty cool.

Quick message to Natalie Bennett, Elisabeth Hendrickson and Dave Liebreich... Jessie done good, y'all should be proud of him. Jessie, if you find yourself down in the  Pivotal Palo Alto office, let me know so we can get together for lunch or something :).

Retro Book Review: Connections

History has the tendency of being seen as static and frozen when we view it from a a later time. What happened is what happened, and nothing else could have happened because, again, at that point, it is set in stone. Once upon a time, however, history could have gone any number of ways, and much of the time, it’s the act of change and transition that help drive history through various eras.

James Burke is one of my favorite historical authors, and I am a big fan of his ideas behind “Connected thought and events”, which makes the case that history is not a series of isolated events, but that events and discoveries coming from previous generations (an even eras) can give rise to new ideas and modes of thinking. In other words, change doesn’t happen in a vacuum, or in the mind of a single solitary genius. Instead it’s the actions and follow-on achievements by a variety of people throughout history that make certain changes in our world possible (from the weaving of silk to the personal computer, or the stirrup to the atomic bomb).

“Connections" is the companion book to the classic BBC series first filmed in the late 70s, with additional series being created up into the 1990s. If you haven’t already seen the Connections series of programs, please do, they are highly entertaining and engaging (ETA: the first series, aired in 1978, is the best of the three). The original print edition of this book had been out of print for some time, but I was overjoyed to discover that there is a current, and updated, paperback version (as well as a Kindle edition) of this book. The kindle version is the one I am basing the review on.

The subtitle of the book and series is "an Alternative View of Change”. Rather than serendipitous forces coming together and “eureka” moments of discovery happening, Burke makes the case that, just as today, invention happens often as a market force determines the benefit and necessity of that invention, with adoption and use stemming from the both the practical and cultural needs of the community. From there, refinements and other markets often determine how ideas from one area can impact development of other areas. Disparate examples like finance, accounting, cartography, metallurgy, mechanics, water power and automation are not separate disciplines, but rely heavily on each other and the inter-connectedness of these disciplines over time.

The book starts with an explanation of the Northeastern Blackout of 1965, as a away to draw attention to the fact that we live in a remarkably interdependent world today. We are not only the beneficiaries of technologies gifts, but in many ways, we are also at the mercy of them. Technology is wonderful, until it breaks down. At that point, many of the systems that we rely heavily on, when they stop working, can make our lives not just sub-optimal, but dangerous.

Connections uses examples stretching all the way back to Roman Times and the ensuing “Dark Ages”. Burke contends that they were never “really dark”, and makes the case of communication being enabled through Bishop to Bishop Post to show that many of the institutions defined in Roman times continued on unabated. Life did became much more local when the over-seeing and overarching power of a huge government state had ended. The pace of change and the needs of change were not so paramount on this local scale, and thus, many of the engineering marvels of the Roman Empire were not so much “lost” (aqueducts and large scale paved roads) but that they just weren’t needed on the scale that the Romans used them. Still, even in the localized world of the early Middle Ages, change happened, and changes from one area often led to changes in other areas.

Bottom Line:

This program changed the way I look at the world, and taught me to look at the causal movers as more than just single moments, or single people, but as a continuum that allows ideas to be connected to other ideas. Is Burke’s premise a certainty? No, but he make a very compelling case, and the connections from one era to another are certainly both credible and reasonable. There is a lot of detail thrown at the reader, and many of those details may seem tangential, but he always manages to come back and show how some arcane development in an isolated location, perhaps centuries ago, came to be a key component in out technologically advanced lives, and how it played a part in our current subordination to technology today. Regardless of the facts, figure and pictures (and there are indeed a lot of them), Connections is a wonderful ride. If you are as much of a fan of history as I am, then pretty much anything James Burke has written will prove to be worthwhile. Connections is his grand thesis, and it’s the concept that is most directly tied to him. This book shows very clearly why that is.

Adventures in Context: What Can A Water Bill Tell Us?

I've posted a few of these in the past, and I have to give my family credit, they put up with a lot from me, but last night gave me a chance to explain to my kids a bit about the exploratory testing process, proposing ideas, and setting up experiments.

What could prompt such a discussion, and why would I bring such a thing up? This time, it was one of the first things that I heard walking into the house. after the "Hi, how was your day, we all did…" Christina looked at me with a less than thrilled expression, and said "we got hit with another HUGE water bill!"

At times like this, there's basically three things one can do. We can go into a fetal position (metaphorically), we can irrationally rant about the unfairness of it all… or if you live with a tester, you might go through something like this (which, roughly, is how last night played out)...

So, let's take a look at the bill… looking at previous months, yes, we definitely are considerably higher. This bill represents July and August, which was summer, with most of us at home or in the house for a much greater period than usual. That might be a good gauge. Let's think of the places we use water… showers, washing dishes, watering plants, bathroom breaks, getting ready in the morning/evening, cooking, plus three fish tanks, not to mention laundry, cleaning, washing cars... it can certainly add up, but does it add up to this?! In previous years, we've had spikes during the summer, but this looks unusually high.

It's usually about this point that I might normally make a variety or pronouncements and recommendations, but I know they will last about two days and then everything will slide back to where we were before. Hmmm, what if we did something different?

"Hey, who'd like to try an experiment with me?"

Truth be told, to some members of my family (especially my kids), this is the most dreaded statement any of them can hear. It has even more weight than "you're grounded". The reason? They know that, when I say "let's try an experiment" I tend to hold them accountable, and they actually have to do things and face hard facts with cold, uncaring data starting back at them. I intend to actually see if they can learn about their real behaviors, not just their imagined ideals of what they might be.

Me? I'm a huge fan of "X-diaries", with the X being whatever it is I want to measure. In this case, water consumption. What does a typical person like me do in the course of the day, and how much water do I consume? Only one way to find out for sure, and that's get into a mode of "write down every water interaction I can think of, and measure it, if I possibly can, or make a rough estimate otherwise.

Little experiments like this typically have me reaching for a notepad, pen and a stopwatch or timer. The reasoning for this is that I want to see if I can spot patterns, or interesting aspects I might take for granted or not notice in my everyday usage. One example is the shower. I turn on the water, and wait for it to be warm enough, and then get in. I don't take long showers because, frankly, I don't have the regular hair maintenance of normal people (of course, I spent more time in front of a mirror with a razor and shave cream, so it kinda evens out for time, but water wise, quite a bit less ;) ). Still, even at the best and fastest practical shower that I can take, the water still needs to warm up, and that takes about 45 seconds. 

Hmmm…

What if I were to take a bucket, and see how much water flowed out of the shower head in that 45 seconds? Better yet, what if I then used that water afterwards to water the plants? Over the course of a week, how much water would that represent? Could it allow me to turn the automatic sprinklers back one minute per zone? Could we use the water for something else, like for rinsing dishes after meals?

It's about this point in these conversations that I am either hit with MEGO (My Eyes Glaze Over) from my kids, or given a litany of reasons why it couldn't possibly work, or how it was a waste of time and effort, etc. Typically, at this point, I also tell them that I have very similar conversations with fellow testers and with programmers as well. In this case, though, I'm not talking about water distribution and use, but about software and how I test it. With this example, I showed them a little glimpse into how I often formulate tests.

First, I need a problem. Something that either is surprising, or comes across as out of the ordinary. A high water pill can be easily compared to web site performance or some other form of interaction that causes someone to say "wait, that can't be right".

Often, we only have hunches to go on, so we try our best to see if we can gather data, even if the data we gather is from a very small area. Reviewing HTTP headers and response times through Chrome tools is very similar to capturing the water in a bucket while it goes from cold to warm.

Once we have some data that we can look at, we can make some hypotheses and make a basis for further interactions or experiments. They may prove to be right, or they could prove to be wrong. If I perform elaborate and detailed captures of water and then transfer it to other uses, I may see total water usage go down. I may also not see any appreciable difference in water usage at all. That might mean that I am just transferring the use in equilibrium… or maybe there's a deeper problem.

Last year, as we were trying similar experiments, we discovered that a sprinkler valve had rotted over time, and that a slow leak had appeared How long the leak had been happening, we had no idea, but we fixed the valve, and for a time, that seemed to make a big difference in water usage. It's entirely possible that we may have a similar situation happening now (our house is just shy of 60 years old, so anything is possible ;) ). 

The point I wanted to show them, though, was that we can use our testing methodology and apply it to a host or problems in our everyday sphere, and likewise, similar challenges in our everyday lives can help inform our testing. My kids often laugh and say "Dad can turn anything into a testing problem". They weren't too amused last night, though I certainly was.  I'm not quite sure if these little talks of mine with my kids will encourage them to test, or will cause them to run with all power and haste to any career where they don't have to hear this crazy testing stuff. Time will tell I guess ;).

An Odd Little Lesson and Life's Little Reminders

My son recently had a birthday, and as a present, my Mom and Dad gave him their older model 1997 Buick Park Avenue Ultra sedan as a vehicle for him to drive. It's a far nicer car than I ever had as a teenager, let's just put it that way.

However, we had some unusual problems with it as soon as we took ownership of it. Each morning, we'd come out and we'd try to start the car... nothing. Wouldn't turn over. We charged up the battery from a jump from my car, started it, drove it around for a bit, turned off the car, came back a few minutes later, restarted it, and it started fine. Next morning, again, nothing.


Hmm... maybe it's the battery. Called my Dad, asked him about how old the battery was. He said probably about six or seven years, so possibly the battery could be old and needed to be replaced.


We did.


All was good, we drove it for a awhile, brought it home... next morning, go to start the car... nothing.


What the...?!!!


At this point, I was about to leave for a conference in Madison, Wisconsin, so I didn't have time to look at it. We called and we made arrangements with our friendly neighborhood mechanic for a thorough inspection of the electrical system when I got back. As I was away, though, I called my Dad, a bit frustrated, and wondering what was going on. In this process, while we were talking, we both remembered something interesting.

Way back, as a kid, my grandfather had a 1963 Oldsmobile Ninety-Eight (actually, my dad still has this car). A classic old school vehicle that was long, wide and just plain massive. I remember several things about that car, but the one that I always though was most amusing was the fact that you could pull the key out of the ignition while the car was running, and the fact that it had an option to "back tick" and turn everything on in the car when the car was "off". Of course, that "back tick" was dangerous because, well, it was easy to pull the keys out and think the car was "turned off" when it really wasn't....


Hang on...


I went out to the car, did yet another "long charge" from jumper cables from my own car. started the vehicle up, and then started to play with the key. Lo and behold, it came out of the ignition after the car was started and running.


Hmmm...


Was it possible that there was a "back tick" option as well? I turned off the car, and rocked the ignition back one more click. Sure enough all of the electric items in the car came to life, and as I pulled out the key, they kept running. Since I kept the radio on to test this, that gave me the confirmation I needed. This 1997 Buick Sedan, at least as far as the ignition and electrical system, behaves exactly like the 1963 Oldsmobile Ninety-Eight.


With that knowledge, I needed to figure out a way to "key" my son in on this. Turns out, a sharpie marker on the steering column did the trick. Now he knows to look and see if the notch in the key housing lines up with the Sharpie mark. If so,  then he knows the vehicle is turned off. With that, we have since had no surprises in the mornings. Car starts, all works fine.


This was a great little reminder that, sometimes, things we remember from long ago may help inform our actions here and now. Had I and my Dad not remembered that old behavior of my grandfathers Oldsmobile, we might not have thought to check for that. Since my Dad remembered it so well (in fact, still deals with it because he still drives the 1963 Oldsmobile and until recently drove the 1997 Buick), he never gave it a second thought. He instinctively knew what to do to prevent that issue. As for me, having not personally driven a GM car regularly since I lived with my parents (well over 20 years ago), I likewise might have never thought of it, save that time many years ago that my grandfather said "hey, here's a neat little trick with this car!"


Next time you find yourself testing something, and seeing a behavior that seems odd, but likewise strangely familiar, do some digging back into the memory banks. It's possible you might find a clue in something you learned, saw or remembered long ago.

AST Re-Election: Do I Deserve Your Vote?

This is going to be very much a niche post. For many, this won't make a lot of sense. For others, it will make a great deal of sense, and it's all of you that it will make sense to that I am hoping will read and consider this post.

Members of the Association for Software Testing, CAST 2013 will be starting just under two weeks from today. I will be there. I will be speaking, helping out where I can, and, I hope, acting as a positive representative of this organization.

This year also marks the end of my inaugural term as a member of the Board of Directors. I am up for re-election. I have considered why I should run for the Board again, and why I should ask each and every one of you to give me your support and your vote.

Short version:

I am running once again, and I am asking for your vote of support for a second term.

Longer Version:

I have enjoyed interacting with all of you over the past two years. I've served in the role of being the "BBST Headmaster",  keeping track of the books for the organization as the Association's Treasurer, and working as the Chair of the Education Special Interest Group, most notably with the group of dedicated individuals who have helped compile and curate the materials that are developing to be the basis of educational modules for SummerQAmp.

I'll be blunt. I'm not an academic. I'm not a consultant. I'm a software tester. A practitioner. I'm an everyday, regular person who works for a regular company. I'm also one who wants to see our profession, craft, discipline, "call it what you will" grow, develop and flourish.

I enjoy being actively involved in those efforts, and being part of the Board of Directors allows me the opportunity to try and see these initiatives, and others, be pushed forward. There are long time projects that need updating, and delivery options that we may want to consider, as an organization, to enhance or replace what we currently use. Frankly, I think I'm just crazy enough to take a number of these on and make them happen.

As a practitioner, I'm also a bit of a cynic (it comes with the territory of being a software tester). I know some of you are thinking... "OK, so what's in it for you?" That's a totally fair question. The fact is, I get opportunities to learn about software testing education in avenues I would not were I not part of this Board. I get the chance to help see grant opportunities develop for international testing conferences, and help see them get funded. I get a lot of satisfaction out of knowing that we as an organization are making a difference around the world, spreading a message of "sapient testing" that breaks free of old methods and default "factory paradigms" that, frankly, don't make sense in an ever adapting digital world.

Yes, I want to see those opportunities continue. I want to champion excellent software testing. I want to get involved with more initiatives that will help make software testing education more available, better performing, more engaging, and yes, dare I say it... FUN! There are many avenues where that could happen,  but I like the mission and purpose of the Association for Software Testing, and everyone involved in it as members. I could approach these opportunities in a number of places, but I'd rather approach it here, with all of you.

Really, that's all I have to say. The final vote is yours. I have enjoyed very much these past two years working with you, and on your behalf. I'd like to ask you for two more years... membership willing :).

Getting Beyond The Slogans

This is part of what I will likely be working into a future talk. That talk, by the way will be in a place I'm not quite at liberty to discuss just yet, but once I've been given the go ahead and confirmation to shout it from the rooftops, rest assured, I will.

So, what does this odd slogan have to do with anything? Keep reading ;).

A few months ago, I wrote a blog post for Zephyr titled "Let's Stop Faking It", and in that post, I gave some thoughts as to how we could get out of this mindset. Initially, I said there were four things we could do:

1. Declare a “Not Faking It” Policy (stand up and admit you don't know what you're doing)

2. Start Looking For Ways to Remove That Ignorance (figure out how to fill the gaps)

3. Find People to “Jam” With (look for people with skill to emulate)

4. Do Not Be Afraid to Ask “Stupid Questions” (which, really goes with #2)

As I thought about how I might expand on this idea, I came up with another thought I wanted to add... eliminate needless jingoism, or more to the point, be "less partisan, more observant".

This bumper sticker comes from a site that I adore, and they specialize in Nuanced, Ambivalent, or Guarded Stickers. This is one of them, and yes, it's the wording that I love. they have several others that are equally as good, but it's the point that the sticker makes that I want to emphasize with this post today. When we make ourselves partisan, we miss a lot of what's going on around us.

First, a bit about my personality. I am rabidly non-partisan. It's not that I'm not willing to "stand for something" (I absolutely am), I just refuse to do so blindly. I'm not much of a tub thumper when it comes to getting into people's faces for causes... well, except for snowboarding; that's something I love with irrational abandon, and I'll extol its virtues until I'm blue in the face. However, if you like to ski, I'll say "great, let's go hit the hill and make a day of it!"

When we allow ourselves to be wrapped up in sloganeering, or what I call the "Home Team" mantle, we are doing so in a way that cuts us off from really learning something. When I was learning Ruby, I did everything in my power to ignore any framework or structure that didn't use it. That meant that, if I wanted to use a language to do something, I forced myself to focus on Ruby. There's nothing wrong with that on the face of things, but in truth, if we do it all the time, we stifle our understanding. When I moved over from Sidereel to Socialtext, I left an environment that was Ruby centic and came to one that was mostly based on Perl. OK, now what is that Ruby foundation going to do for me? Surprisingly little... and surprisingly, a lot! Now, I'm looking at code modules that are written in Perl, and sometimes I think "oh, this would be so much less of an issue in Ruby", but sometimes I also notice "Hey, that's kinda cool... Ruby doesn't do that!" In short, letting go of the jingoism, it lets me see other possibilities that a "rigid mind" wouldn't allow. 

I will admit, I can be a "fan" of things as much as the next guy, and I'm not saying "don't be a fan". What I am saying is "don't let your fandom get in the way of how you see things".  Don't let your enthusiasm bias you to what could happen if you were more detached, or less "emotionally invested". If you're not willing to shut it down completely, OK, but make it a mental exercise to try. Think of ways you could approach a problem and remove your "favorite thing" as the default option. Just for a while. Try to interact a different way. After doing so, do some introspection.What did you learn in the process? I'm guessing quite a bit. It may be positive, it may be negative. Either way, it'll probably get you closer to actually understanding what you want to know than if you just grab for the default preferred option. 

Onboarding the "New Tester"

Our little team is growing! What started as a single person covering all of software testing has grown into a small team of senior software testers, with additional support from contractors and interns. Additionally, we also had another Senior QA Engineer come on board last week. I was asked to shadow him and be his pair and mentor so that we get everything up to speed.

Having gone through a lot of these situations myself, and having taken copious notes, I figured we'd be on Easy Street in no time. I'd just show him where I stubbed my toes, and show him how to avoid them.

We always say that we will make it smooth for the "new tester" because we've already walked the path, and we'll show them the way. The problem is, we know the path, and we've laid out what we believe to be good trail markers, but in many ways, the path we have laid out is really only good for us. As I complained earlier about the "implicit knowledge" the team has, unless I'm going to spell out to the very most intimate minutiae of what needs to be done, yes, there's a lot of implicit knowledge that I'm expecting people to have and understand.

Just because I know backwards and forwards the steps to a process doesn't mean that the obvious things to me are going to be obvious to them. Sure enough, after a few fits and starts, we ironed out the issues, and we got everything set up and finally running. The process taught me that things that I know are important, and are plain and obvious guide markers to me, may be totally missed by someone else, not because they aren't smart or accomplished at what they do, but because their expectations and experiences might be slightly different than mine.

When we get the chance to bring someone new on board, let's try to resist the temptation to give the bare basics and then let them fend for themselves. Instead, let's walk them through areas we discovered pitfalls, and explain why they are pitfalls. We need to be clear as to exactly what areas might cause problems or be difficult to get set up correctly. We need to let them know that big complex systems require a lot of knowledge of their moving parts, and that there's no way they're going to absorb it all in a week or two. Heck, I've been here almost five months and I still have areas that I am exploring, mapping and trying to understand. Remember how it feels to be the new tester on the team... then act accordingly. It will save them heartache and it will also save your sanity.

WAVEing for Accessibility

Through the past several weeks and months, I've been getting much more familiar with Accessibility testing and some of the areas that we should be aware of when we do this kind of testing. Most of my efforts have been focused on trying to get two primary applications to work with our product. These tools are JAWS (a screen reader for visually impaired users) and Dragon (a speech processing application for those who may have physical limitations working with a standard keyboard and mouse).

As i was trying to look to see if our product was working, much of my time was spent manually using JAWS and Dragon, and then pointing out to the developers where we weren't providing enough information. During this process, the lazy bones in me kept saying "seriously, isn't there something that we can use that can speed this up? Do we really have to listed to or talk to the page to see what it's set up to do?"

Thankfully, the answer is "yes, there is something we can use that can allow us to check accessibility aspects of pages." There are actually several, but one quick, easy and free way is to use WAVE.

What WAVE lets users is quickly analyze areas that might be problematic from an accessibility standpoint, and allow the user to preview them based on the importance of those issues. For example, WAVE did a quick analysis of TESTHEAD and found the following issues:

A more detailed inspection gives us the following:

In addition to errors, it can also show you what you are doing "right" with your pages:

Ever wonder how your site looks when its styling is turned off? In general, we don't like to think about this in this day and age, but it's col to get a quick glimpse... is our site organized in a way that is genuinely useful if all the eye candy disappears?

Curious to know how your site fares in a simple contrast test? For the record, TESTHEAD fares terribly. Much as I like the design, it's giving me pause and making me reconsider my aesthetic conceits.

The examples above use the WAVE site. You can also bring the exploration closer to home with the  WAVE toolbar (currently only available for Firefox). Here's how I look from the "Errors, Features and Alerts" perspective.

Ever wondered where the tab key will take you in your pages? I never used to think about this, but since I work with JAWS regularly, I think about it a lot more. Pretty sad to think that, if you want to highlight my Blog link, you'd need to hit 58 tabs to get there.

Text only mode? Heh, that's actually OK:

Curious as to what all of the possible icons and options mean, as well as a glossary of potential accessibility issues? WAVE's got you covered:

I realize for many of you out there, this may be of very peripheral interest, but these past few months have opened my eyes as to some of the challenges we face when we try to make software accessible to as many users as possible. A lot of the site design prevalent today is very difficult to navigate by using alternative means. If you are not willing, or able, to take a plunge into the world of screen readers and speech recognition directly, but you are curious to see how your site stacks up against some accessibility standards, get WAVE and give it a try. You will learn a lot about your site(s), in ways you never imagined.

Weekend Testing Americas: Passing the Baton

Tomorrow will be an interesting day for me, not so much by what I will be doing, but for what I will not be doing.

So far, only once in Weekend Testing Americas history have I physically not been part of a session, and that was because I had just been released from the hospital with a broken leg. Albert Gareev gamely stepped up and took my place for that session, and I'm grateful to him for doing that.

This time, while I have something else I need to do, as I was considering postponing the session until I was available, Justin Rohrman contacted me and said he had a session plan, and wanted to know if he could run with it. I realized that, if I was actually going to have others be facilitators, I had to actually let them facilitate. Thus, rather than wait and postpone until I could be available, I've asked Justin to take the lead and facilitate tomorrows Weekend Testing Americas session completely in my stead. Just as Ajay and Markus had to finally decide I could lead sessions on my own, now is the time to for me to put my faith in Justin and let him lead this one on his own.

I'm sure he'll do fine :).

I shared some thoughts with Justin, and I realized that a lot of the advice I gave him was appropriate to any other testing endeavor, and wasn't just specific to our Weekend Testing environment. With that, I offer the following.

A Facilitator for Weekend Testing events is really responsible for just three things. They need to manage time. They need to articulate the session goals. They need to encourage participants to get involved. That's the core of what we do. The rest (setting up sessions, promoting them, adding people to the Skype chat, typing up an experience report, etc.) that's all mechanics and "busy work". It's important, don't get me wrong, but the first three items really are the most important. Manage the clock. Articulate the goals. Encourage participation and effort. Any test manager in any organization is really responsible for the same things.

Managing time: This is important because we have a two hour block of time. We could do longer, I'm sure, but then it cuts into people's other commitments, and really, we're already asking a lot to get testers to willingly give up part of their weekend as it is (don't get me started on the testers in India that join us at 12:30 a.m. for these sessions... yes, that's after midnight their time. That puts a whole new spin on dedication in my book :) ). Thus, it's vital that we keep a focus on the time that we have, and that we accomplish what we set out to do in that short time period with focus and clarity. That leads to...

Articulate the goals: I remember well a couple of years ago when James Bach attended a session and he, rightly, called me out for our sessions being too vague. Since we had such a short time together, it was vital that we were able to make missions and charters that were clear, specific, and targeted to specific tasks. I was worried that such targeted missions and charters would be too limiting. He proved to me that, in most cases, that's not the case. Even with just testing one simple area of a site, we were able to have a conversation that spanned almost two hours of active dialog, and that was with us looking at just one screen. It wasn't that the screen was any different than any other, but that the goal was focused and specific, so that we were able to discuss and consider the aspects of that one screen without distraction or loss of direction. It was a worthwhile lesson, and one I try to practice still.

Encourage participants: Some people are just not comfortable with talking out loud, even in chat clients. There may be many reasons for this. Some may be novices, some may have language barriers, and some might not want to be seen to "appear the fool". I've learned to never underestimate the terror of that last one. For many, they fear that they will say something foolish, or appear to be "less smart" among their peers. For this purpose, I often put my own ignorance on display in these sessions. Yes, I'm facilitating, but that doesn't mean I'm an expert on the topic. Often, I know less than most of the participants... and that's OK. My goal isn't to be the one saying everything, it's to encourage dialog and learning. I can do that best by asking about things I genuinely don't know, and often, that's a lot. By my doing so, in many cases, it helps others open up and ask questions, too. We're not here because we are experts. We're here because we all can learn from each other, and that's the whole point of Weekend Testing. We learn from each other, sometimes in small ways, sometimes in much larger ways.  As long as we each learn something to bring to our craft, that's what matters.

Tomorrow is a new day. A new facilitator will step up and run with the session, and I will not be there. I do, however, want to encourage all who can be there to do so. Details on tomorrow's session can be found here.

Some Realizations of Being a "Host Family"

First, I want to start this by saying that the past week that I have had has been one of the most rewarding experiences ever, and I mean that sincerely. I greatly enjoyed opening our home to two of the eight middle school children that came to visit us here in our town from Japan as part of our Sister City Exchange program.

This was precipitated by my older daughter's desire to be a part of the group to go to Japan this year, and one of the expectations is to have the family of a U.S. participant act as a host family for the Japanese participants. We were happy to do so, and in the process, invited two of the sweetest girls on Earth to stay with us for a week. All of that is fantastic. Now, on to some things I discovered.

1. Do not assume that something that is done where you are matches what is done elsewhere.

I learned this by having our participants have dinner with us at a somewhat high end sushi bar (well, high end for where we live). In the process of having that dinner with them, I discovered that most Japanese do not combine foods like tempura, sukiyaki and sushi in the same meal. I also discovered that the salad's that we often get to start a meal are also not particularly "Japanese", it's an American thing. Another interesting discovery, "American Sushi" uses a lot more wasabi in its making. Not as a garnish on the side, but actually inside of the sushi itself. The girls bit into several nigiri pieces, winced and say "wow, this is too strong!" I then watched them dissect the nigiri pieces and scrape away the wasabi. I found this fascinating; what I had often taken for granted as just the way things were done wasn't. In fact, it was an adaptation for our local environment that I had become accustomed to. Now I know better :).

2. Factor in at least an hour plus for any activity that is done with a group.

We took a trip to the city (San Francisco) and jumped on a MUNI line to go out to the Marin Headlands, the primary goal being to walk across the Golden Gate Bridge, and then go explore some other areas of San Francisco. In the process, we gathered up all of the participants and all of the host families to come along. That worked out to about 30 participants. When  you are bringing thirty people along to anything, odds are that you will need to pad significant time to make sure everyone can be accommodated. Bathroom breaks easily take 20 minutes with a group that size. Snack stops? Thirty minutes. Meal breaks? Well over an hour, plus an hour or more at any given venue. Needless to say, there were a lot of things that we did that ended up taking a lot of time because of so many people. The easiest way to get some leverage? Split up the group. We were able to see a lot more once we split off into smaller sections and only had to manage twelve people instead of thirty.

3. There are some things that are, on the surface, universal.

Early teenage girls of just about every culture will squeal with delight at a cute bunny rabbit. Cats are adorable. So are dogs. Spiders and snakes tend to weird them out.  Cupcakes are a hit. A trip to the Apple Store and letting them test out headphones will last all night if you let them. Late night giggle fests are a norm. With this, my girls and these two girls from Japan got along famously. The Japanese girls considerably better English than my daughters spoke Japanese, but they were able to make themselves understood for hours, and it seems that Americas Funniest Home Videos translate well in to any language.

4. Being a Host Family is rewarding and exhausting
The participants had a full itinerary each day, and both my wife and I participated in it. It left little time for relaxation and just sitting still. Net result, we are both very tired, but it's that super accomplished kind of tired, the kind that makes you feel like you want to pass out, but if you do, you will do it with a grin from ear to ear. One other thing to remember, if you decide to be a host family, any other project you might have going on, just park it. I promise, you will not have the time or the emotional energy to do much of anything else.

5. It's possible to have one's heart stolen in a short amount of time.

Six days ago, I didn't know anything about these eight Japanese students. Today, I sent them home with a heavy heart. It was such a short time, but I already miss them. I know my girls miss them terribly; they cried and hugged for the longest time this morning, until they had to be almost forcibly separated (well, OK, that's a little over dramatic, but it was a hard goodbye). Just six days, and I grew to care about both of them tremendously, as though they were my own daughters. That will fade with time, I know, but the good thing is that the seed for a lifelong friendship has been planted. I will encourage my girls to  help keep it growing, if for no other reason than that Christina and I want to know how they are doing.

For those who have similar opportunities, if there is any way you can be a host family to such a program, do it! It's such an awesome experience, and I would gladly do it again. I have a sneaking suspicion that once more may be in our future, when our youngest is in eight grade. If that happens, I'll gladly sign up to do it again.

A Different Stress Test: My Wife as Tour Guide

This may seem an odd title, but as you all know, I can find ways of making a testing allegory to just about anything, if given a chance.

Last weekend, my wife, Christina,  and I decided to take a weekend away. Our kids are old enough now that they can look after themselves for a night, and we wanted to celebrate our 20th anniversary (albeit delayed) in a more "festive" location and manner, so early Saturday morning, we boarded a plane, and took a short flight to Las Vegas.

First, a couple of things to know about us. Neither of us drink, we're not "partiers", and neither of us gambles or frequents strip clubs, etc. Some might say that, with that kind of a list (or anti-list), Las Vegas would be the most boring place on Earth. Not so. Christina considers Las Vegas to be a second home, especially the Strip with all its glitzy hotels, resorts and amenities available for all ages. She's kind of an expert on it. Ask her to point out the ten best shows on the strip and she can tell you without hesitation. Best restaurants? She can tell you dozens off the top of her head. Architectural designs unique to each hotel? Yep, she knows all of them and has combed just about every inch of every hotel from Mandalay Bay to the Stratosphere (there's a few she hasn't been to yet, but rest assured, she'll get to them soon enough :) ).

My point is, if anyone wanted to consult a "domain expert" on the places to see and things to do in Las Vegas that didn't revolve around drinking, gambling, or strip clubs, Christina is it. Yet even with that, we discovered that you can throw curve balls even to the best of experts and make them have to adapt.

A little context might help. Christina and her mom have been going to Las Vegas once or twice yearly as long as I have known them. I think this tradition started when Christina was a teenager, and without fail, at least once a year (sometimes twice), exploratory trips are planned, conducted, and travelogues of sorts are reported. Typically, they go mid week and they go "off season", usually meaning February or October. In that situation, they have plenty of time to go where they want to go, do what they want to do, and see what they want to see. For our trip, however, timing just happened to be that we were going to go on a weekend, and in particular, St. Patrick's Day weekend.

The net result was that we discovered the stress test that was getting from our hotel (at the Palazzo) to our dinner reservation (at THEMix at THEHotel at Mandalay Bay) and then to our show (Blue Man Group at Monte Carlo) and then negotiate how to get back to the Palazzo.

First, a bit about the Las Vegas Strip. If you look at it from a map, it looks like everything is close together. In reality, it's not. The hotels are just so big that they look like they are next door. The distance from Mandala Bay to the Stratosphere (the traditional boundaries of "The Strip") is about six miles. To get from place to place, cabs are important. On this night, we discovered that the queue to get a cab from the Palazzo was about 300 people thick. WOW! Not something we had anticipated. We did get our cab, and we did get to our dinner reservation... at which we had to wait a bit to be seated, and wait even longer to get our dinner (almost an hour... was it because I ordered Wagyu?).

From there, we made our way down and, caught another cab to get us to the Monte Carlo for our show (with a little time to spare, but not much). After the Blue Man Group Show (which was amazing, btw, and totally worthy of its own post :) ), we walked out of the hotel hoping to catch a cab... and saw the queue was triple the size it was at the Palazzo. Doing the math and realizing how long it would take, we decided to walk and have Christina play "tour guide" to me and show me her favorite spots. However, even this was stymied because of the sheer volume of people everywhere we went. Just moving from place to place was like salmon swimming upstream. Even with all of that, it was a lot of fun watching Christina, even under pressure, modify her approach and where we would go, and the angles she'd choose to take to avoid the crowds when possible. In short, she could adapt quickly because she knew just about every angle of the Las Vegas Strip there was. Were I to try to figure it out on my own, well, we probably would have made it back, but a lot later, I'd bet.

So what's my testing analogy? We may think we know an area, and we may think we understand how something works, but we don't really know how well or what parameters we know until we put some stress on the system. The fact is, pathways that are well known and easy to navigate may become considerably less so when our environment changes, and on this night, changes radically. If you have done your homework in advance (in this case, about two decades worth) you can adapt and move more freely. Likewise, when a system is under stress, it's vital we know as many avenues to get around as possible to allow us to see where people might go and interact with. Quiet times are good times to explore and find them, but rest assured, under load and stress, even quiet areas of an app will take on more load and pressure, and the results may be less than ideal.

One bright point... the odds of you running into several revelers puking in the bushes... probably higher in Las Vegas the night before St. Patrick's Day than in your app... but you never know ;).

When Two Wrongs Make a Wall

You've probably all been through this before. A target date has been set. You've run a product through its paces. You've tested thoroughly, and had multiple members of your team look over something for sanity's sake. All is good. You know it. You feel it, let's push this and call it a day.

The next morning, you walk in, think to yourself "let's just do a couple little tweaks. That package that we need to update, seems like now would be a good time to do that". Update completed. Now that that's out of the way, I'll show our PM what's been deployed. Only now, instead of that clean, seamless environment we'd seen, verified, tested within an inch of our lives, now it's showing weird behavior, oddly placed elements, and all in the room are having that classic "WTF?" reaction.

This is truly one of the worst experiences to have, and all of us testers have had to deal with it at one time or another. So what causes this? As I've been discussing this with our intern, I described it as the idea of "two screens". For this to make sense, think of the classic "shoji screens" used in Japan to separate rooms, made with wooden frames and using rice paper as the wall material. In one screen there is a number of holes. In the second screen, there is also a number of holes but placed in a different pattern. Lay them on top of each other, and for the most part, you will not be able to tell that there is anything but a "solid wall" in front of you. Remove either of the two screen layers, and the holes become visible, and obviously so.

After a bit of poking around, we realized that, yep, updating a particular module that our site uses made for a change in how some of our code was being called. The issue was always there, but it had been masked by the behavior of the previous revision third party module. By updating that module, it "fixed" something, and by making that fix, it showed a flaw that had been there for quite some time, but only now, with this update, were we able to see it.

I mention this because, in our rapidly changing and intertwined web world that we live in today (never mind mobile, which adds its own interesting "screening effects"), the interaction of components, even at the most trivial levels, could open up a vast area of unexplored options. Is it practical to review every change when third party components are updated? Probably not, but if the areas you are currently testing make use of those items, even peripherally, make a point of reading about and understanding the updates being performed, even if they seem to be relatively trivial.

Much of the time, the component changes will have no effect at all on what you are developing and testing. Sometimes, though, a change results in removing a mask, and that change will make it possible to see things you didn't see before. Much better to make those discoveries while testing, rather than in front of your PM when you want to have something signed off. In short, pay attention to the updates that happen around you. You might find out that those solid walls are not so solid after all ;).

ExploreIt Book Launch Party: An SF Selenium Meetup

This is a day that I've been waiting to see happen for quite awhile. For those who may remember, I started reading and covering ExploreIt! during the Summer of 2012. Through several beta revisions, I've seen the book grow and develop, and today it is now officially available in print form as well as e-book format. This isn't going to be a book review... I already wrote that ;). Nope, in this case, I'm here to discuss what Elisabeth will be discussing, which will be "The Top 5 Reasons Your Team Needs Exploratory Testing".

For those not familiar with Elisabeth and her background, Elisabeth is a tester, programmer and agile advocate. She believes strongly that Exploratory Testing is more necessary now than it was five or ten years ago.  One of the points that she puts forth early on is that "Users do not (and often will not) complacently follow the path you set for them". In short, "you just can't trust your users to use your system the way that you hope they will use it". They might, but frankly, users are often unintentionally creative. We think we control the file system. We think we control the database. We even think we can control the user... we don't. In fact, the areas we really do have control over is very small. Thus, it's critical that we actually explore these areas that we can't trust.

Another point Elisabeth makes is that "no one can reason through all of the possible parameters of a change".  Also, she makes the point that "sometimes things just don't connect the way you think they do". Additionally, the biggest goal and reason why exploratory testing is more relevant today than ever is that, if teams actually use these exploration tools, we will "increase the odds that if there is a problem, you will see it".

Short and sweet, direct and to the point, and now it's time to draw for books (ten of them, in fact). Must be present to win, and I WON ONE! YAY :)!!! Since I have all of the previous beta copies, I think I will give this to the test team over at Socialtext, so that everyone on my team can see the stuff I have been trying to evangelize all these years...

And with that, it's time for cake. Catch you all in a bit :).

Does it Really Do What You Think It Does?

Yesterday I had one of those interesting experiences. When I say "interesting", that's often a euphemism for "disquieting discovery that I need to do something better". Yeah, that's exactly what I mean by it.

Case in point. We had a convoluted story with a lot of test examples we had to go through. We raked this sucker over the coals, and we found all sorts of stuff that was fixed, refactored and, by one of our developer's admission, "it took me longer to explain what we did than it took me to write the whole thing". It's a good bet that, in those circumstances, you may find that there are holes. I was prepared for holes. I wasn't prepared for the question I did receive, though...

"So, how can I make a clear and specific choice that shows the effect of this change, and in a way that I can explain it to a customer?"

It seemed so simple, such a clear and easy thing to do... and yet, I realized that I was struggling with this, because our Quality Director, as he walked me through this particular scenario, kept showing me things that would effectively get in the way of doing what we wanted to demonstrate (browsers with the way they cache and store items, the differences between browsers, etc.). Each time, I was starting to feel my confidence fade a little bit. I went from thinking I was very confident that we were on good ground with our testing to feeling rather uneasy. What did we miss?

It wasn't what we missed, but what we didn't articulate. The developer and I did so much back and forth with the story that we, at the time, knew we'd run the story through its paces, multiple times. What I didn't do was give enough of an indication of what, specifically, could be told to our support engineers so that, in the event of explaining the story changes, they would easily be able to explain the pros and cons to our customers.

I mention this because, sometimes, there's a danger we'll lose the balance between being lean and fast, with "just enough documentation" to be effective, and explaining enough so that those who see stories after we have finished them have the right amount of information to likewise be effective. The key takeaway from this experience was to say "can you, in a short but succinct way, explain to anyone who reads this story days, weeks or months after the fact, exactly what it is the story is accomplishing, and how to, on the system, show exactly what a change is doing? If you can do that, then you can go lean and mean with documentation. If you can't, then adding more and more documentation will probably not make the story any more understandable".  This hearkens back to my comments a few days ago about "implicit instructions". We know what they are, the developer may know what they are, and the Product Manager may also know what they are. If, however  any key individual in that line and beyond does not know, then there's a problem, and it can be exacerbated the farther away from the implicit understanding we get. It just goes to show that even veteran testers may have to do a gut check every now and again to make sure that they can simply and directly explain what something does, and be sure  that it really does do what we think it does.

Same As It Ever Was?

According to the Internet, and Mayan prophecy, December 21, 2012 is supposed to be the end of the world. I figure, if this proves to not be true, this will be something to allow me to celebrate a wonderful year of testing. If it does prove to be true, well, no one will be here to read this, so I suggest reading this quickly ;).

2012 was a pivotal year for me, in that I had a chance to do many things I'd never done, I had a chance to participate in a number of unique opportunities, and I made some decisions that have really made me question if it made sense to do things the same way I've done them for so long. Also, for those astute musical nerds out there, I'm referencing Talking Heads "Once in A Lifetime" once again with the title. It's proven to be quite a versatile song for these posts over the past few years.

When I wrote the first of these recaps in 2010, I was preparing to leave a job I had worked at for almost six years, and very much looking forward to a new adventure in a new capacity. In 2011, I shared many of the lessons I'd learned from making that step, and how being involved in the broader community had become very important to me. Thus I find it interesting that, here at the end of 2012, I am writing this message at yet another company, at the start of yet another "excellent adventure". "Same as it ever was?" seemed rather fitting, as this was not a year of business as usual. Not by a long shot!

2012 was a year of travel and outreach, and the opportunity to learn about and work with a number of interesting initiatives. I concluded my dive into Ruby and learning as much of the language as "Learn Ruby the Hard Way" would inspire me to do. This was a project started in 2011, and it ran for three months. I learned a lot along the way, and grew to appreciate many of the nuances of Ruby and how it works. My personal library of Ruby titles is huge now, which is a little ironic since, in my new role at Socialtext, I am looking at code that written mostly in... Perl :). Some might comment that I've wasted my time with all this Ruby focus, but I don't think so at all. What I've been able to do is approach a language at a deeper level than I ever have in the past, and do so with the eye of sharing my experience with others. That helped me internalize a lot more of it. Don't get me wrong, I'm not what I would consider a great programmer, nor even a moderately good one. Still, there's a level of appreciation and achievement I'm quite proud of, and I feel it will help me look at other languages and be a little less intimidated.

My friends Lynn McKee and Nancy Kelln invited me to participate in the POST 2012 workshop up in Calgary, Alberta, Canada, back in March. I was the facilitator for this event, and help the participants present their topics, discuss their ideas and critiques, and I also had a chance to present the idea of Weekend Testing as a service that any company could incorporate and associate with its test teams. An aded bonus, Lynn took me to Sunshine Village, one of the ski resorts in Banff National Park. Yep, I got to tick off a bucket list event... I snowboarded in Canada!

I had the pleasure of going to New Orleans to help plan out many of the events that would be part of the CAST conference for 2012, and visit some of the areas of the city (Bourbon Street, Royal Street, etc.) that I'd only heard stories about. It also gave me a chance to participate in the first Workshop day of STP-CON Spring 2012, likewise in New Orleans. I had the experience of being able to go and participate in and live blog five different workshops, and participate alongside many other intelligent and engaged testers.

I presented my first full paper and presentation at a conference this year. It was originally to be at PNSQC 2011, but a broken leg sidelined me and I was unable to present it. A friend who felt bad that I couldn't give my talk contacted Lee Copleland and suggested my talk would be a good fit for their conference. Lee read the paper and decided "yes, we'd like to see this presented" and offered me a spot on the program at STAR EAST 2012 in Orlando, Florida. I acepted and presented my talk. Additionally, I won Best Paper for "Delivering Quality, One Weekend at a Time". For the record, that was a seriously cool experience!

Weekend Testing in the Americas had a more regular schedule at one session per month. As we held our sessions throughout the year, we noticed an interesting pattern. There is a group of regular attendees that often participate. We have a number of brand new testers that come on and try it out a few times, then disappear. We have a number of one-offs, those who try it and never come back. Because of this, I determined it would be a good idea to get some additional brains into the mix to help develop sessions, content ideas, and other areas of focus. Albert Gareev has continued to be a great help to me in this regard, and we welcomed JeanAnn Harrison and Dan Gold into the mix of regularly contributing facilitators. Seriously, thank you to all of you, it made this year's sessions more enjoyable, and it gave me peace of mind to know that, if I couldn't be there for a session, that it would happen and all would be well. Additionally, I appreciate the influx of fresh ideas and different approaches.

Early in 2012, I had a chance to answer an article written about the SummerQAmp program, and what it hoped to accomplish. That proved to be a fateful message, in that I started a collaboration with the organizers of SummerQAmp and, along with members of the Association for Software Testing and other interested test professionals, we started writing what we hope will become a complete and practical introduction to the world of software testing. We delivered several modules and ran a beta test of the materials with a number of interns, and the response was "This is great! Can we have more of this?!" The answer, I hope, will be "Yes", and make no mistake, that will be a primary focus for me and for the AST EdSig in the new year. If you'd like to participate, please drop me a line!

I was invited to participate in Test Coach Camp, which happened the weekend prior to CAST 2012. This was an open-space conference event, where a number of testers participated and presented a variety of topics. I had the chance to present three different sessions (mentoring interns, teaching leadership skills, and a systematic deconstruction of Weekend Testing and the question "if we rebuilt it, what would you like to see us do?"). CAST 2012 also was the first chance to present the idea that has been my focus for much of the year, looking for that elusive balance between Test Driven Development, GUI Automation and Exploratory Testing. This topic showed up in a number of formats this year, and each time I approached it, I learned something new. First, it was a paper submission for PNSQC, then an emerging topics talk at CAST, then a full presentation at Agilistry Studios, and finally as a poster paper presentation that I gave (dozens of times) at PNSQC.

2012 saw me branch out and start contributing articles to a number of different outlets. Thanks to  ST&QA, Testing Planet, Atlassian and Zephyr for allowing me the opportunity to write for a broader audience, and for their giving me a chance to open this blog up to more testers and people interested in my writing. This year I also set a record for traffic with a post that is now number 1 with a bullet on TESTHEAD. Which post? Learning to Tell Different Stories, where I compared the storytelling tradition in Japan to what us "westerners" are used to, and how the differences and nuances open us up to asking different questions once we see and understand that there are different ways of seing things beyond our own world view. I also enjoyed participating in ST&QA's "Ask  the Tester", where I had the chance to answer a number of questions from the broader testing community. Also, I was a presenter in the Agile Transitions Online Conference for Software Test Professionals, where I presented my talk on "Being a Lone Tester on an Agile Team".

TWiST had another year of great conversations, great participation, and crossing the 100 episode mark (as of this week, we're up to episode #127). I always think of the old television maxim that, for a show to live on forever, it needs to pass 100 episodes to be eligible for life in syndication. I'm not sure if that's applicable for a podcast, but it's great to see that there is an appreciative audience, and that we can bring these discussions and ideas to you each week. I also enjoyed the various panels I participated in, and the shows I could contribute my ideas and thoughts to various discussions. Finally, I would be remiss were I not to say thank you to Justin Rohrman and Mark Tomlinson, who stepped in this year to help me edit episodes and do some of the "grunt work" that goes into getting these shows ready to be packaged and released. Seriously, your help is greatly appreciated!

During 2012, I continued my active involvement with the Miagi-do School of Software Testing, where Matt and Markus decided that I had earned the right to be advanced to a Black Belt Level Instructor. It's both gratifying and humbling to be associated with so many great testers, and while I now have the title of Instructor, sometimes I wonder who the real teacher is. I feel like I learn more from those I interact with than they likely learn from me.

As I have taken on the role as Chair of the Education Special Interest Group within the Association for Software Testing, I made the decision to step out of an active teaching role for the time being. While I will still be teaching some classes, I wanted to focus this year on giving others the opportunity to step up and learn how to lead the BBST classes and encourage those who haven't had the chance to assist and get a chance to teach as well. My goal for 2012 was to broaden our instructor pool, and that will continue to be a primary goal for 2013.

Hanging up my Lone Tester status was definitely not something I could have foreseen earlier this year, but looking at the interactions with others in so many other mediums, perhaps I should have seen it as inevitable. I decided that through all of the interactions I have had with my fellow testers, and with some feelings of frustration with my role as a lone tester, that I would put out some feelers and see if there were some test teams that would be interested in having a "Veteran of the Psychic Wars" join them. I have to admit I was surprised that so many responded, and so quickly. Thus, with the chance to "practice what I preach" regarding interaction, engagement and peer involvement, I made the decision to make the move from Sidereel, where I was a Lone Gun, to Socialtext, where I now work with a small but focused team of four testers... and by the way, we're looking for another tester to join us after the new year, so if you're interested (and local ;) ), let me know.

So many people have made this an amazing year for me, and to mention everyone by name will likely mean I'll leave someone out, so if I do, please don't feel slighted (and hey, if you do, email me and I'll put you in... blogs are cool like that :) ). Cheers and much appreciation to Aaron Scott, Albert Gareev, Anne-Marie Charrett, Becky Fiedler, Ben Simo, Benjamin Yaroch, Bill Baker, Catherine Karena, Cem Kaner, Dan Gold, Dee Ann Pizzica, Doug Hoffman, Elisabeth Hendrickson, Francis Adanza, James Bach, Janette Rovansek, JeanAnn Harrison, Jeff "Toxic" Burchell, Jon Bach, Justin Rohrman, Keith Klain, Ken Pier, Kevin Haggard, Lee Copeland, Lynn McKee, Mark Tomlinson, Markus Gaertner, Marlena Compton, Matt Barcomb, Matt Heusser, Mimi Mendenhall, Nancy Kelln, Patti Swift, Pete Walen, Peter "Pantera" Arzhintar, Rich Szeto, Rick Baucom, Scott Barber, Shampa Bannerjee, Thomas Ponnet, Timothy Coulter, and Zach Larson. Thank you for challenging me, for making me question my ideas, my motives, and my goals. Thank you for helping me make it possible to make changes, take burdens off of my shoulders and help me so that initiatives I started are being shepherded and able to keep going. Thank you for what has honestly been, at least as far as software testing is concerned, my greatest year (and remember, I said the same thing last year, and the year before that).

Oh, and should the world not end on December 21, 2012, then let me suggest that we follow the wise advice of Abraham Lincoln, who said:

"Be excellent to each other. And... PARTY ON, DUDES!"

I'll Be the Roadie...

As my son gets older and takes on more interesting challenges, it's been interesting learning a bit how his mind works and what interests him. Currently, he has an interest in film production.

This past weekend, he told me that he needed to do some work for his final in Film, and he decided he wanted to do something where he interviewed street performers in San Francisco, specifically the stretch between Fisherman's Wharf and Pier 39. Seeing as I really only have a handful of these opportunities left (he's a Junior in high school now; three more semesters and he'll be graduated and off to college), I wanted to help him meet his objectives.

At first, there was some resistance. "Dad, it's OK, I don't need help, I can do this on my own!" Well, of course you can, dude, but that's not why I'm asking to come along. I wanted to see if there were ways I could see how he works and understand what he does, and in the process, be of some assistance to him (and sure, keep an eye on him, just a little... I'm a Dad, cut me some slack here ;) ). I finally swayed him with some simple words. "You handle the vision and the talking. I'll be your roadie." Meaning, I'll carry all of the gear for you, and you focus on finding the people you want to film and talking to them. He liked the sound of that, and thus, we were off to the City.

Seeing as it was a cold, foggy Saturday (typical for San Francisco) and expected to rain (not as typical) we were concerned that there might not be enough people to interview in the time we had allotted, so I offered to do some advanced scouting with him. As we walked towards Fisherman's Wharf, he noticed a Rastafarian bongo player setting up with a coffee can for tips. My son walked up to him and asked him if he could film him and use him in his project. No answer. Stone silence but him playing his drums. I waited to see how my son would react, and what he would do. Would he ask again? Would he move on? Do something else? I realized this was a different level of communication for my son than he was used to, and decided that this was a time that a "Seasoned roadie" could offer some assistance. I reached in my pocket, took out a few dollar bills, placed them in the drummer's coffee can and asked him the same question. He lit up a bit and said "sure, what would you like to know?" With that, we set up the microphone and camera, and Nick did a quick and somewhat stilted interview with the drummer. Not real communicative, but that's OK, we have to start somewhere.

As we walked down Fisherman's wharf several times, we noticed there weren't any performers out. Was it the weather? Were we just early? What could we do as a contingency plan. It was here I suggested to my son that, maybe, talking to some people on the street about their experiences with street performers might prove to be interesting. It would offer a different perspective to his project. He thought about it for a minute and said, "sure, let's give that a try". We set up the gear again, and we waited. Most people just walked past us hurriedly, but a couple of people did stop to talk with us. After  about ten minutes, he decided we should take a look farther down the wharf and back to Pier 39.

When we made it back to Pier 39, there was a performer with a Chapman Stick. I lit up at this, since I was very familiar with the instrument, and maybe my fan boy-ness got the better of me, but I suggested that this would be a great person to talk to. Since he was also set up in a prime spot right outside of Pier 39, I figured he was likely a professional who rented the space (turns out I was correct with that assumption). I also figured, if I could strike up a conversation with him about his instrument and what I already knew about it, that might work in our favor, too. I mentioned how much it blew me away when I first heard Tony Levin playing the Chapman Stick in the early 80's with King Crimson, and of curse the performer was familiar and we talked about other musicians who likewise used the instrument. thus, when we asked if we could interview him for my son's project, he was very gracious and let us record several numbers and he provided a lot of great interview commentary.

As we walked a bit further back towards Fishermans' Wharf, I noticed a lady and a gentleman get out of their car on a side street in Dickens' era garb. Performers? Certainly. Street performers? Hmmm, maybe not, they were near the Cannery, and that has a pretty swank hotel in it. My guess is they were performers for the hotel by  the level of their outfits, but I figured, hey, they're here, let's see if they'll talk to us. My son agreed, and we went over to chat with them. Turns out they were waiting for another member of their group, and yes, they worked for the hotel, but they were happy to talk to us about what they did, and why they enjoyed performing here particularly, and when their third compatriot arrived, they sang a song for my son to film.

As we were feeling pretty flush with the excitement of capturing two good interview subjects, we walked back to the wharf and noticed an elderly man dressed like he was from the Roaring Twenties performing a magic act on a portable table. MY son looked at me, winked and said "let me handle this one!" He went up and talked to the man and asked if he'd be willing to be interviewed. the magician said sure, but not for free. I casually handed my son $10, and motioned to the box on the table. He put in the $10 and then the magician said he'd be happy to perform his entire act for him, provided he'd be willing to send him a copy of the footage (sure thing!).

After filming this, it was starting to rain, and we figured that this was going to be it for the day. As we were walking back to the car, my son saw, across the street, one of the guys he really wanted to interview... the elusive and legendary "Bush Man". This guy is famous (and infamous) for scaring and freaking out tourists, and he would have definitely made for a fun part of the project, but with the rain coming down and not wanting to risk damaging his equipment, he decided to let it go.

All was not lost, however, since just around the corner, under a protective awning, one of the "Gold Statue Dancers" was there. For those not familiar with the "Statue dancers", they typically stand perfectly still until someone steps up and drops some money into their cup, and then they start dancing in a robotic fashion. This one stood on top of a milk crate, and incorporated the crate into the performance, actually sliding and stepping with it (quite proficiently, I might add). At first I was leery, thinking this guy wouldn't want to be interviewed, but my son, having seen how the rest of the day had gone, decided to give it a try. He walked up, dropped a $5 into the dancers cup and asked if he could ask him some questions. Gold Statue Dancer Man turned out to be the most animated and entertaining of all the interviews, as well as the longest.

As we drove home, I told him a bit about how today was a good example of what I do as a tester every day. With him as my product owner (my customer) I acted as roadie and helped set up the gear, helped evaluate situations, offered suggestions of avenues to try, provided support and alternative approaches when things weren't panning out, and provided some domain knowledge to help him communicate with the performers where needed. Mostly, though, I provided him with information that may or may not have been effective in helping him make decisions about what to do. All in all it was a successful outing, and a god reminder to me that "playing the roadie" can be a good metaphor for good testing, both in software and in film production.

Have I Found What I'm Looking For?

As those who follow my blog might notice, I tent to mangle pop culture references to make my titles or make other post points when I write here. Today is no exception.

As many of you may be aware, I started a new job on Monday, November 19, 2012. You might also notice that this blog has been quiet since Tuesday, November 20, 2012. That is not a coincidence! Much of my time the past two weeks, with the exception of Thanksgiving weekend and some much needed time with my family, I've been dealing with a single minded focus; coming to grips with the paradigm shift I've undertaken.

For years, I was the lone gun. I was the guy who either had the answers, or needed to get the answers. I was mostly responsible to myself, and mostly took care of things on my own. Today, I'm the polar opposite of that world view. I'm part of a testing team with some rather talented people, and we communicate about our findings... a lot. Socialtext uses a tool called Colloquy, which is, effectively, an IRC client. There's two main chats that go on all day, every day. One's for dev, the other is for testing. When I say these conversations go on all day, that's exactly what I mean. We all contribute, we all share what we see, we all keep each other updated as to what we see and what we find, and those chats inform where we go next, what we do next, and what the developers see next. So I'm on that window and I'm typing a fair amount each day... and I came to a realization as I was doing it.

There's somewhere else that I do this. Once a month, for a couple of hours, I and a number of other testers get together, via a chat mechanism, and we talk testing. We pick an app, we explore it, we find issues, we talk about them, we decide where to go next based on what we learn, and we discuss what we learn. If what I'm describing sounds a lot like Weekend Testing, that's because it is what I'm describing. A couple of years ago I lamented "oh, wouldn't it be really cool if an organization actually used the Weekend Testing model? Think of what we could all learn! Think of what we could share! Think of how much wasted time could be avoided because we are actually communicating!"

For the past few years, I've dreamed of seeing something like this exist. I've hoped, but had my dreams either dashed by indifference, or the feeling that there's no time to do such a thing. I often wondered, though, what would a company look like that actually did this? Now I know. It looks like Socialtext. It looks like that because they actually leverage what they learn and they use it together. If something is going off the rails, we all know pretty quickly. If something looks promising, we're encouraged to explore it. Explore and share what we find. Feedback is almost immediate, and it's refreshing, exhilarating, infuriating, exhausting, maddening, and enlightening, all at the same time. It's the chance to live the Weekend Testing model and ethos full time. The funny thing is, though, it's left me with little energy to do many other things, at least for the near term.

So, for those wondering why I've been so quiet as of late, there's the reason. Also, be careful what you wish for... you just might get it :).

TESTHEAD REDUX: Testing's "Lone Gunfighters"

Image URL: The Old Gunfighter

As I was first trying to design TESTHEAD and figure out what would be the core point, I played with a lot of different ideas. What could I really offer to the community? What would be my niche? What could I speak to differently than anyone else out there? There were a lot of us talking about changes to how we test, to looking at Agile practices, to integrating development and testing, and touching on testing automation, but I wasn't seeing the way I could add any kind of a fresh voice to those discussions. About a week in, though, I came across what was, really, the one thing that I could speak to very directly, and in a lot of different ways. That topic was what it feels like to be the sole tester in an organization.

Before I standardized on the term "The Lone Tester" (yep, it kinda' sounds like The Lone Ranger, so I ultimately went with that slightly poetic moniker) I played with lots of different terms and names. Regardless of what I called myself, though, the reality was the same. For all practical purposes, I was alone in my testing efforts. How did that shape me then? How does it shape me now?

THEN:

When the situation of the “Lone Gunfighter” happens, testing takes on an interesting hue. Now, it’s all on you, and when you are the last line of defense, you really are all there is between the product getting out into the wild and bad things potentially happening. Sobering is a good word for this situation.

NOW:

I'd say the big difference between then and now is the fact that, for all intents and purposes, I've put down the cudgel of "it's all on me". It may seem that way, but it's really not. In an Agile organization, there are other people testing, not just me. My role is that of the sole person whose primary responsibility is testing. For everyone else, it's a peripheral activity, but I've stopped believing that I'm the only one actively testing in the organization. What's more, I've dropped the conceit that I'm the only one qualified to test. That's an insult to the programmers, who to be honest can do quite solid testing, and think of things I don't.

THEN:

Many times, we can feel like we have no direction, or that we are being barked at to get this or that accomplished, with little to no support from others. By putting yourself into the mode of consultant, you change the relationship. Now you are providing a service to a customer, and in this case, the development team and the ones purchasing or using the product are your customers. When the development team becomes a customer and your goal is to provide top notch service to that costumer, your entire mindset and focus changes.

NOW:

I still see a lot of truth in that mindset, more so than I did when I first wrote about it, because in many ways, in the organizations I've been part of, I've often been held apart from the programming team. I'm not entirely sure if that was by design or by circumstance, but there is a sense that there are a lot of people who believe that testing should not become too "chummy" with the programmers. By being held as separate, like a service provider to the team, that "respectful distance" is maintained. Again, I've heard that many have gone beyond that relationship and have had a more casual connection, but that hasn't quite been my experience. Not yet anyway :).

THEN:

Make sure that you define your role as clearly as possible, and what you feel the expectations for your contribution should be, and let them say what they feel it should be as well. This agreement may be formal and in writing, or it may be something discussed in meetings or between team mates. Either way, get it out there so everyone knows the expectation and can work to meet or exceed it.

NOW:

I agree with this more now than I did then. This can doom you if you get it wrong, or be your biggest ally if you get it right.What's also important is to regularly revisit this topic. Situations change, and different people have different opinions and attitudes. I had a period where I had three directors and all three directors had a different idea what my contribution should be. That's totally OK, but it also reinforces why it's important to consistently get those who you are working with to come to a consensus as to what your role and contribution should be, and what the team actually values. One director thought that my developing technical chops and programming skills would be the best use of my time. Another thought exploratory testing was of much higher priority. Both are valuable, both are important, but different people put different weight on certain things. Knowing that and working with that in mind can be a big help.

THEN:

Get over the us vs. them mentality: If you have no idea what this means, congratulations, you work at a company that “gets it”. If you are all too familiar with this mentality, make the first steps to change that dynamic. Testing and development are allies, they both have the same goal, to ship a product that has high quality and that will meet the needs of its customers. No other attitude is going to help that other than development and testing working together to help solve problems.

NOW:

Nothing to add to it, I believe this 100%, now and then.

THEN:

Accept that you will not be an expert in everything: You will have knowledge gaps, and at times those gaps will be huge, but you have to make a clear assessment of your strengths and weaknesses and put them out there. Yes, make them known, the strengths and the weaknesses. Also, make a game plan to overcome those weaknesses where possible and telegraph the fact that you are working to close those gaps.

NOW:

How ironic that I was talking then about my knowledge gaps, only to step into an environment where there were ten times more moving parts! Again, a Lone Tester will not know everything, they will not be able to be all things to all people on the team, unless they are an especially rare kind of superhero. I'm not that person.  That need not be a barrier to success, but it needs to be addressed seriously and honestly. If you have big goals of making huge strides in a particular domain, know you will have to devote a substantial amount of "private time" to make that happen (meaning off the clock). On the clock, don't be surprised if you find yourself juggling five balls at once, all day, every day.

THEN:

When you are Lone Gunfighter, you may be able to find someone who can help you in the development group, but often they may have limitations as well (especially around testing questions and issues). Reach out to other testers you have worked with and ask questions. I still have mentoring relationships with friends who were instrumental in my development over the past 20 years, and I also occasionally get asked questions as to something I have experience in. I believe it’s important to have a mentor and to be a mentor, so look for opportunities where this can be utilized. 

NOW:

Twitter and the testing blogosphere is the single biggest resource of ideas, inspiration, cheer-leading, coaching and mentoring that a Lone tester could hope for. I follow around 400 people that are involved with testing or are thought leaders in software testing and software development. So many breakthroughs with ideas and approaches have happened through discussions that take place on Twitter (especially those that I just read between other testers) that I feel I can go there and be "spiritually fed" daily. That's not a saccharine sentiment, I actually mean that. I'm amazed at the discussions and insights I have learned from other testers all over the world.


Post Script:

The irony to writing this update is that, as of last Friday, November 9, 2012, I made a decision to, for now, hang up my Lone Tester status. That will deserve a much more detailed blog post, and I will be writing that soon enough. Many of the thrills of that line of testing are also many of its more draining challenges, and for me personally, some of my reflections on these ideas and the way that things have been for the past decade has made me desire the interaction of a testing team, where I can interact with and bounce ideas off of other testers. Not just testers in the Twitter/blogoshere world, but testers I'm sitting with, interacting with, mentoring and being mentored by them. To that end, I'll be starting a new adventure with SocialText on Monday, November 19, 2012.