How I am Overcoming "Expectational Debt"

A phrase I have personally grown to love, and at times dread, is the one that Merlin Mann and Dan Benjamin coined in 2011 during their Back to Work podcast. That phrase is "expectational debt". Put simply, it's the act of "writing checks your person can't cash" (there's a more colorful metaphor that can be and is often used for this, but I think you understand what I mean ;) ).

Expectational Debt is tricky, because it is very easy to get into, and it is almost entirely emotional in nature. Financial debt is when you have to borrow money to purchase something you want or need, and it invariably involves a financial contract, or in the simplest sense a "personal agreement" where the money owed will be paid back. It's very tangible. Technical debt is what happen in software all the time, when we have to make a shortcut or compromise on making something work in the ideal way so that we can get a product out the door, with the idea that we will "fix it later". Again, it's also tangible, in the sense that we can see the work we need to do. Expectational debt, on the other hand, is almost entirely emotional. It's associated with a promise, a goal, or a desire to do something. Sometimes that desire is public, sometimes it is private. In all cases, it's a commitment of the mind, and a commitment of time and attention.

I know full well how easy it is to get myself into Expectational Debt, and I can do it surprisingly quickly. People often joke that I have a complete inability to say "no". That's not entirely true, but it's close enough to reality that I don't protest. I enjoy learning new things and trying new experiences, so I am often willing to jump in and say "yeah, that's awesome, I can do that!" With just those words, I am creating an expectational debt, a promise to do something in the future that I fully intend to fulfill, but I have not done the necessary footwork or put the time in to fully understand what I am taking on. Human beings, in general, do this all the time. We also frequently underestimate or overestimate how much these expectations matter to other people. Something we've agreed to do could be of great importance to others, or of minor importance. It's also possible that we ourselves are the only ones who consider that expectation to be valuable. Regardless of the "weight" of the expectation, they all take up space in our head, and every one of them puts a drag on our effectiveness.

Before I offer my solution, I need to say that this is very much something I'm currently struggling with. These suggestions are what I am doing now to rein in my expectational debt. It's entirely possible these approaches will be abandoned by yours truly in the future, or determined not to work. As of now, they're helping, so I'm going to share them.

Identify your "Commitments"

Get a stack of note cards, or use an electronic note file, or take a 365 day single day calendar, whatever method you prefer, and sit down and write out every promise you have made to yourself and to others that you have every intention of fulfilling. Don't just do this for things in your professional life, do this for everything you intend to do (family goals, personal goals, exercise, home repairs, car repairs, social obligations, work goals, personal progress initiatives, literally everything you want to accomplish).

Categorize Your "Commitments"

Once you have done this, put them in a priority order. I like to use the four quadrants approach Steven Covey suggests in "Seven Habits of Highly Effective People". Those quadrants are labeled as follows:

I. Urgent and Important

II. Important but Not Urgent

III. Urgent but Not Important

IV. Not Urgent and Not Important

My guess is that, after you have done this, you will have a few items that are in the I category (Urgent and Important), possibly a few in the III category (Urgent but Not Important), and most will fall into Categories II and IV.

Redouble or Abandon Your "Commitments"

These are questions you need to ask yourself for each commitment:

- Why do I think it belongs here?
- Will it be of great benefit to me or others if I accomplish the goal?
- Is it really my responsibility to do this?
- If I were to not do this, what would happen?

This will help you determine very quickly where each of the items fall. Most expectational debt will, again, fall in categories II and IV.

For your sanity, as soon as you identify something that falls into Category IV (Not Urgent and Not Important), tell yourself "I will not be doing this" and make it visible. Yes, make a list of the things you will NOT be doing.

Next is to look at the items that fall into Category III. These are Urgent but Not Important, or perhaps a better way to put this is that they are Urgent to someone else (and likely Important). They may not be important to you, but another person's anxiety about it, and their visible distress, is making it Urgent to you. It's time for a conversation or two. You have to decide now if you are going to commit time and attention to this, and figure out why you should. Is it because you feel obligated? Is it because it will solve a problem for someone else? Is it because you're really the only one who can deal with the situation? All of these need to be spelled out, and in most cases, they should be handled with a mind to train up someone else to do them, so that you can get out of the situation.

The great majority of things you'll want to do, and want to commit to, will fall in Category II. They are Important, but they are not Urgent (if they were Urgent and Important, you'd be doing them... probably RIGHT NOW!!!). Lose weight for the summer. Learn a new programming language. Discover and become proficient with a new tool. Plan a vacation for next year. Read a long anticipated book. Play a much anticipated video game. These are all items that will, in some way, give us satisfaction, help us move forward and progress on something, or otherwise benefit us, but they don't need to be done "right now". Your goal here is to start scoping out the time to do each of these, and give it a quantifiable space in your reality. I believe in scheduling items that I want to make progress on. In some cases, getting a friendly "accountability partner" to check in on me to make sure I'm doing what I need to do is a huge incentive. A common tactic that I am using now is to allocate four hours for any "endeavor space". I also allocate four hours in case I need to "change tracks" and take care of something else. This may seem like overkill (and often, it is), but it's a shorthand I use so I don't over-commit or underestimate how long it will take to do something. Even with this model, I still underestimate a lot of things, but with experience I get a bit better each time.

This of course leaves the last area (Category I), the Urgent and Important. Usually, it's a crisis. It's where everything else ends up getting bagged for a bit. If you ever find yourself in an automobile accident, and you are injured, guess what, your getting treatment and recovery rockets to Category I. In a less dire circumstance, if you are the network operations person for your company and your network goes down, for the duration of that outage getting the network to work is Category I.

I hate making promises I can't keep, but the truth is, I do it all the time. We all do, usually in small ways. Unless we are pathological liars, we don't intend to get into this situation, but sometimes, yes, the expectations we place on ourselves, or the promises we make to others, grow out of proportion to what we can actually accomplish. Take the time to jettison those things that you will not do. Clear them from your mind. Make exit plans for the things you'd really rather not do, if there is a way to do so. Commit to scheduling those items that provide the greatest benefit, and if at all possible, do what you can to not get into crisis situations. Trust me, your overall sanity will be greatly enhanced, and what's more, you'll start to develop the discipline to grow an expectation surplus. I'm working towards that goal, in any event ;).

Adventures in Context: What Can A Water Bill Tell Us?

I've posted a few of these in the past, and I have to give my family credit, they put up with a lot from me, but last night gave me a chance to explain to my kids a bit about the exploratory testing process, proposing ideas, and setting up experiments.

What could prompt such a discussion, and why would I bring such a thing up? This time, it was one of the first things that I heard walking into the house. after the "Hi, how was your day, we all did…" Christina looked at me with a less than thrilled expression, and said "we got hit with another HUGE water bill!"

At times like this, there's basically three things one can do. We can go into a fetal position (metaphorically), we can irrationally rant about the unfairness of it all… or if you live with a tester, you might go through something like this (which, roughly, is how last night played out)...

So, let's take a look at the bill… looking at previous months, yes, we definitely are considerably higher. This bill represents July and August, which was summer, with most of us at home or in the house for a much greater period than usual. That might be a good gauge. Let's think of the places we use water… showers, washing dishes, watering plants, bathroom breaks, getting ready in the morning/evening, cooking, plus three fish tanks, not to mention laundry, cleaning, washing cars... it can certainly add up, but does it add up to this?! In previous years, we've had spikes during the summer, but this looks unusually high.

It's usually about this point that I might normally make a variety or pronouncements and recommendations, but I know they will last about two days and then everything will slide back to where we were before. Hmmm, what if we did something different?

"Hey, who'd like to try an experiment with me?"

Truth be told, to some members of my family (especially my kids), this is the most dreaded statement any of them can hear. It has even more weight than "you're grounded". The reason? They know that, when I say "let's try an experiment" I tend to hold them accountable, and they actually have to do things and face hard facts with cold, uncaring data starting back at them. I intend to actually see if they can learn about their real behaviors, not just their imagined ideals of what they might be.

Me? I'm a huge fan of "X-diaries", with the X being whatever it is I want to measure. In this case, water consumption. What does a typical person like me do in the course of the day, and how much water do I consume? Only one way to find out for sure, and that's get into a mode of "write down every water interaction I can think of, and measure it, if I possibly can, or make a rough estimate otherwise.

Little experiments like this typically have me reaching for a notepad, pen and a stopwatch or timer. The reasoning for this is that I want to see if I can spot patterns, or interesting aspects I might take for granted or not notice in my everyday usage. One example is the shower. I turn on the water, and wait for it to be warm enough, and then get in. I don't take long showers because, frankly, I don't have the regular hair maintenance of normal people (of course, I spent more time in front of a mirror with a razor and shave cream, so it kinda evens out for time, but water wise, quite a bit less ;) ). Still, even at the best and fastest practical shower that I can take, the water still needs to warm up, and that takes about 45 seconds. 

Hmmm…

What if I were to take a bucket, and see how much water flowed out of the shower head in that 45 seconds? Better yet, what if I then used that water afterwards to water the plants? Over the course of a week, how much water would that represent? Could it allow me to turn the automatic sprinklers back one minute per zone? Could we use the water for something else, like for rinsing dishes after meals?

It's about this point in these conversations that I am either hit with MEGO (My Eyes Glaze Over) from my kids, or given a litany of reasons why it couldn't possibly work, or how it was a waste of time and effort, etc. Typically, at this point, I also tell them that I have very similar conversations with fellow testers and with programmers as well. In this case, though, I'm not talking about water distribution and use, but about software and how I test it. With this example, I showed them a little glimpse into how I often formulate tests.

First, I need a problem. Something that either is surprising, or comes across as out of the ordinary. A high water pill can be easily compared to web site performance or some other form of interaction that causes someone to say "wait, that can't be right".

Often, we only have hunches to go on, so we try our best to see if we can gather data, even if the data we gather is from a very small area. Reviewing HTTP headers and response times through Chrome tools is very similar to capturing the water in a bucket while it goes from cold to warm.

Once we have some data that we can look at, we can make some hypotheses and make a basis for further interactions or experiments. They may prove to be right, or they could prove to be wrong. If I perform elaborate and detailed captures of water and then transfer it to other uses, I may see total water usage go down. I may also not see any appreciable difference in water usage at all. That might mean that I am just transferring the use in equilibrium… or maybe there's a deeper problem.

Last year, as we were trying similar experiments, we discovered that a sprinkler valve had rotted over time, and that a slow leak had appeared How long the leak had been happening, we had no idea, but we fixed the valve, and for a time, that seemed to make a big difference in water usage. It's entirely possible that we may have a similar situation happening now (our house is just shy of 60 years old, so anything is possible ;) ). 

The point I wanted to show them, though, was that we can use our testing methodology and apply it to a host or problems in our everyday sphere, and likewise, similar challenges in our everyday lives can help inform our testing. My kids often laugh and say "Dad can turn anything into a testing problem". They weren't too amused last night, though I certainly was.  I'm not quite sure if these little talks of mine with my kids will encourage them to test, or will cause them to run with all power and haste to any career where they don't have to hear this crazy testing stuff. Time will tell I guess ;).

Adventures in Context: Talking Commuting With a Newly Minted Driver

Recently, I had a chance to share with my son some ideas and comments about how testing is less about whether something passes or fails, but is about analysis of information and helping to make a qualified decision based on experimentation and feedback.

One of the games I love playing with my kids is "Armchair Economist". I'm a bit of a geek when it comes to experiments, continuous learning, and interesting discussions, as well as ways of looking at problems that might have slightly ambiguous answers. This is one of those times that playing "armchair economist" led to an interesting discussion about testing.

For years, my commute was an easy thing to deal with and calculate. I worked in San Francisco, which meant, basically, two things. Either I did a rail commute, or I drove. Both have their positives and negatives, but for the most part, driving was just not a reasonable solution because traffic and parking prices made it genuinely unpalatable. Since I had a meeting I had to be at each morning that started at the same time, commute decisions were easy. Go to San Bruno Caltrain station, get on train at the appropriate time, get to meeting on time. No fuss. 

When I changed jobs, and made the shift to working in Palo Alto, plus the fact that where I was working was very close to the CalTrain station, again, that was a no-brainer most days (because of the way that our company works with the City of Palo Alto, I do have a parking permit that gives me "free" parking in the Civic Center garage). Sometimes I have things going on that require me to drive in, but most of the time, train still makes the most sense.

Pat of this change-up involved me changing my schedule so I could drive my kids to school. I'd drop them off, and I'd go on to the next station (in Millbrae) rather than back track and go to the one in San Bruno. 

Now that my son has a driver's license, and a car of his own to drive, he's now handling the "driving of the kids to school" routine. He's also become very "price conscious" about how much fuel costs, and more specifically, just how much traveling and mobility that fuel actually gets him. Part of this was prompted by the fact that, since he's received his licence, he would drive me to the train station and drop me off on various days, and as part of this (including being my ride home) he was wondering how much it was costing him to "ferry me to the train station and back". 

This prompted a discussion… "Hey, Dad, how much does your monthly commute cost you?" 

I showed him the price tables, including some of the additional costs of parking, and at first, we both agreed, it looked like going to Millbrae over San Bruno is the better deal… but is it, really?

With that, I said "hey, let's be 'testerly' about this. Let's build a model." 

We sat down and we started with the obvious comparisons. 

I live in San Bruno. 

I work in Palo Alto. 

Caltrain divides its fare rate into zones, instead of an origin/destination price model. 

Travel from San Bruno to Palo Alto covers three transit zones ($179/mo. if you use the Clipper card) 

Travel from Millbrae to Palo Alto cover two transit zones ($126/mo. if you use the Clipper card)

Three zones is more expensive than two zones, $53 per month more. 

OK, so it's cheaper to travel from Millbrae than San Bruno. We're done, right? Well, not so fast…

Let's consider the parking situation. In San Bruno, although it can be hit and miss, and you may need to walk a bit farther on some days compared to others, it's entirely possible to park on Huntington Avenue for free. It's so much more possible that the San Bruno CalTrain lot usually only has a handful of cars in it on any given day. 

Cost for paid Caltrain parking? $5/day, or $50/month with parking permit, doesn't matter the station. 

OK, so what's the option for free parking in Millbrae? Actually, there's very little in the way of nearby/available free parking. The side streets have strict time limits or "no parking" policies. The closest "Free" parking is about a half-mile away. This is reflected in the parking levels in the Millbrae lot. By 9:00 a.m on any given weekday morning, the lot is completely full, out to the furthest parking spots. 

Translation: a $50 premium added to the price of the commute. So the difference now is just $3/month in Millbrae's  favor.

Hmmm… here's a thought. What's the difference between driving to San Bruno Caltrain vs. driving to Millbrae Caltrain? 

San Bruno: 4.2 miles (round trip)
Millbrae: 8.4 miles (round trip)

When I was driving my kids to school, that didn't make as much of a difference, since their high school was close to the Millbrae station, and I'd be going out that way anyway. It didn't make an sense to double back. Now? It's completely my choice, so I'm choosing to drive double the distance. I drive a traditional car, a 2001 Ford Escape 4WD w/ a V6 engine. Nice vehicle for off road play, camping and snowboarding, but it's strictly average when it comes to driving around town, to the tune of about 15-20 miles per gallon. To be conservative, let's use a 15 MPG value. If I were to take an average of 23 days for commuting in a given month, it's 97 miles total per month to the San Bruno station, 193 miles total per month to the Millbrae station. Gas right now is $3.99/gallon, and at 6.44 and 12.88 gallons respectively, the cost for gas (specific to my commute and nothing else) is $25.69 for San Bruno, $51.39 to Millbrae.

So where does that put us?

- San Bruno to Palo Alto Total Cost per Month: $204.69

- Millbrae to Palo Alto Total Cost per Month: $227.39

Net difference? $22.70 per month extra to use Millbrae instead of San Bruno.

-----

This is where I paused, and did my classic testers smirk and said "so, are we done?"

My son said "yeah, it actually costs less to commute from San Bruno, so you should commute from San Bruno."

Testers, I see you smirking (well, i don't see you smirking, but I know you are ;) ).

"Yes, on a pure dollar basis, it would make sense to say "Commute from San Bruno, you'll save money". Is it really that simple, though?" 

My son, to his credit, knows that, whenever I throw out the "Is it really that simple" comment, or something to that effect, that I'm about to lead him on a bit of a Socratic adventure. To his credit, he hasn't gotten to the point of rolling his eyes and walking away when I do this. Today, he didn't disappoint.

"Well, it would make sense if everything were the same, right? If every train stopped at each station, and if the schedule were the same, then the cost savings would make sense… but Caltrain doesn't work like that":

- None of the "baby bullets" stop at San Bruno.
- In fact, there's a lot fewer trains that stop at San Bruno.
- Also, trains that stop at San Bruno are either full stop trains or on limited stops.
- An average commute out of San Bruno to Palo Alto will take about 35-40 minutes (33 minutes being the fastest option on one specific train).
- The fastest commute out of Millbrae will take less than 25 minutes
- Even if we don't catch the baby bullet trains, there's more trains that stop at Millbrae than San Bruno in any given hour, about three times as many.

Here's where I smile a bit, since I see he's considering something. I leaned in and said "ok, so now that you've said that… which is the better value for commuting?" 

As expected, he thought about this for a while, and said "well, it depends on what matters more to you. If you care about the total cost, then commuting from San Bruno (and parking on the street for free) will be the better choice. If you care about flexibility, then commuting from Millbrae would make more sense. The difference is $23, or one dollar a commute day. If, in my schedule, I had to change up my routine for some reason, or needed to get there at a specific time, then sure, it would cost me $5 to park in Millbrae on that day, but I could still do that 4 times and come out ahead. $2.70, in fact. If I had to vary it more than 5 times, then commuting that month out of Millbrae and buying a monthly parking pass would be a better deal."

-----

I stopped the discussion at this point. I told him that, if we wanted to, we could go into serious minutiae and get measurements under various conditions to see total time performance (door to door, how long it took to walk to the platform from where we parked, total time difference on an average vs optimized schedule, etc.) but that this exploration was pretty good in and of itself. It gave us a lot of good information.

What was interesting was how he saw that the dollar amount, which would have been easy to quantify in isolation, masked a lot of nuance of the situation. I explained to him that the nuance is where a real thinking mind needs to make an analysis, a total calculus of the whole situation, and then make a judgment call based on all the information. As he learned, it's not all cut and dry. Additionally, there's a new construction project happening in San Bruno, and the station is going to be changing location in a few months. At that point, the "Free Parking" option may no longer be a viable option, and we'll have to reconsider this whole exercise again at that point.

So, what do you and your kids talk about ;)?

Cascading Fail: A Crash That Hits Close to Home

As many of you are aware, there was an airliner coming from Seoul, Korea that crashed upon landing at San Francisco International Airport Saturday, July 6, 2013. Much has been said in the news about the crash, and much will probably be said in the following days and weeks. My point for this post is not  to talk about the crash, or the myriad of systems that were or were not available. It doesn't change the fact that an airline crashed, scores of people were injured, two girls were killed, and a major International Airport was brought to a grinding halt while the events that followed played out.

This crash had a direct impact on me, though, in two ways. First, it is the cause of my daughter, who has spent the last eight days in Japan, being unable to come home yet. Because of the shut down of the airport, many flights had to be diverted to smaller airports, which were quickly overwhelmed by the increased load. Many flights that were scheduled were cancelled, my daughter's included. Thankfully, due to some herculean and determined efforts by the team of chaperones, as well as the good will and kindness of the city of Narita, Japan, they were able to weather the hiccup. Note, this "hiccup" meant that their return was delayed by three days, including, at the current time, a rerouting through Seattle and an almost 24 hour layover.

The second way that it impacted me is knowing that, for the two girls who were confirmed dead, both were students coming over on an exchange trip from China. Both were mid teenagers. In other words, both were mirroring my own daughter's trip. I am a realist, and I understand Black Swan events, and the likelihood of a repeat performance is way less than a million to one, but that doesn't calm a father who now has new uncertainties and anxieties. Needless to say, it was a little too close to home.

Note, I'm not blaming anyone for this, but in the tester's world that I inhabit, rarely is there such a thing as a "isolated problem". Usually, when something goes wrong, it affects entire systems, and those systems also affect entire systems. The net results of an error could cascade out and cause devastating problems, and the after effects not seen until well after this issue has occurred. Yes, I am one of those people who can see a testing story in everything, but today I am doubly reminded of how early mistakes not caught can ripple out, and we really have no way of knowing just how far the problems  can cascade. We may have a momentary interest, as long as the issue doesn't affect us. Once it does, though, we start to see a much broader world of issues and problems. A good reminder to me in my workaday world to try to find problems early, while the course correction options are much wider and still possible.

Pay it Forward, or Pay Back With Interest

This phrase is the message that greets me every morning when I come into work. It's the headline of our Kanban board, and it's a very simple reminder. Things may take great up front effort, they may be frustrating, painful, and difficult to deal with, but they will invariably be less difficult and less painful than doing something expedient and deciding to "deal with it later".

I had this driven home to me over the course of the past week as I tackled my ever recurring "New Year's Day Project". It's been the same project every year for the past decade or so. On New Year's Day, the Christmas decorations get taken down, put back into storage, and then the process of cleaning out the garage to put everything back occurs. Every New Year's Day it's the same. I wake up, I clean, I sort, I throw away, I organize, I de-clutter, I de-junk, and somewhere around midnight, I crawl into bed with a clean garage and an orderly life... which proceeds to succumb to entropy within a few days, weeks, months, to the point where I have to do it all over again the following year.

This time, I tried something different. I decided to tackle the job early, as in day after Christmas early. Once I knew what we had, and what we were going to use and keep, I went medieval and decided to do my best to downsize as much as humanly possible. Go for broke, get rid of everything that did not have practical utility. Forget the keep/hold/toss cycle, go straight to toss. Do my best to hide the carnage from the wife and kids, because I know if they see what I am doing there will be much weeping, wailing and gnashing of teeth (and in full disclosure, I was caught a few nights ago by my daughter who, as she saw what i was sorting and cataloging for donations, yelled a hearty "Oh no you don't!!!" and rescued a few dozen items. I smiled and said I'd be willing to bet that half of what she rescued would be on the same chopping block next year at this time, but hey, peace at home is a valuable thing, so I'm not going to threaten it over a handful of dolls and toys that she has outgrown but doesn't want to admit just yet.

My point is, this was a heavy lift. It took  lot of time, it took a lot of resources, but it got done, and it made it so that, for the first time in years, I was finished with the garage before midnight, and in a state that might actually be maintainable. Part of the challenge was that, before, I kept shuffling things around, finding new places to put things in the hopes that I would actually use them. While I can't claim that I didn't do some of that this time, I certainly did a lot less of it. Having the time to think, to purge, and to figure out where they actually needed to go makes a world of difference.

So with this start of a New Year, may I suggest smaller steps towards goals rather than making huge jumps. You ultimately get more done, you don't spend as much time and/or energy, and you don't make decisions that will end up making you do even more work later on. Pay it forward, or pay it back, with interest. Needless to say, I'm thinking the former is a much better deal.

"Break Throughs" Sometimes Require "Breaks With"

For the past decade plus, I have had a "beautiful monster" in what is effectively my home office. I say effectively because, in all reality, the Home office is really a place for a book case, a fold down futon/couch, my closet with my clothes and miscellaneous items for said futon (makes for a semi-nice guest bedroom in a pinch), but the dominating feature (and I used that word very literally), is the Computer Hutch/Armoire. It's a beautiful piece of walnut furniture, with six doors, an internal light, adjustable media shelf, places for CPUs, printer, scanner, external drives, filing drawer, and a retractable desk that can be folded in and closed up behind six panels. It's beautiful, and it's huge, as in 24 inches by 48 inches by 72 inches huge, and that's without the desk pulled out.


This was the first purchase I made for my "home office" in 1999. When we bought our house, I was overjoyed to finally have a room that was going to be my office. An actual room, not setting up in a side closet as in our previous place, not shunted out in the garage when I outgrew the closet and the heat from a SPARC station and a 21" CRT monitor became unbearable. This was going to be a real honest to goodness dedicated office, a veritable "man cave" of my very own, and to house the tons of bulky computer equipment I had at the time, including the said SPARC pizza box, 21" CRT monitor, and a secondary PC that was connected to the same monitor and keyboard, the armoire was ideal, along with a big executive chair and everything in its place.


Fast forward 12 years. The SPARC station and original PC CPU are long gone. They have been replaced a couple times. Today, my digital life and work reside on a Macbook Pro and a Toshiba Sattelite PC. The need for a dedicated printer is now in the family room where the entire family can use it. Network connections come via wi-fi. External storage is now more often than not in the cloud, with some DVD backup media for good measure. Future platforms are possibly looking to be tablets (whether iPad or some Android flavor) or smartphone (whether iPhone or Android). The simple fact is, the Armoire, while it looks nice, it's a relic now. It doesn't fit the way that I work any longer, but I've tried to force myself to  keep using it for years now. I paid a lot of money for it. I wanted to get my money's worth out of it.


The funny thing is, we often invest a lot of ourselves, our energy, and our emotions in items that are just plain and simple past their effective use life. Don't get me wrong, this desk will be a gem to someone with the room for it and who would like to have it serve a similar need (it has enough room for a big flat screen TV inside, plus the ability to be a fully functioning office desk if necessary. It's just that it no longer fits my life and needs any longer, and justification to keep it around just burns up cycles, not to mention my having to do clutter control of a major scale just to keep the area around it tidy.


Today, I made a decision. I called my local Salvation Army store and said I had a great piece of furniture looking for a new home. They'll be coming by later this week to pick it up and bring it to their showroom. In return, I will have 48 cubic feet of space to do something with. I may set up a card table or something to have a place to sit and type if I need to, or I may just leave the space empty and actually have the ability to walk into the room and sit down and breathe and relax. I may re-purpose the room entirely, who knows? The fact is, I can do my work anywhere now. I don't really need a dedicated office any longer, and I'm willing to bet that developments in the coming years will make the need for a dedicated space even less relevant (though I will probably need it as a quiet space to do show production for the foreseeable future).


At the end of this year, think about the tools, practices, policies or equipment that you have "invested in" and ask yourself, are these items, practices or policies genuinely useful to me, or do I keep them on life support just because I'm sort of attached to them? If you can say that you really do use them and they really bring value to what you do, then you'd be perfectly fine to leave things be and not do anything. However, if you find, like me, that you are spending a lot of energy producing "churn" for no other reason than to churn, give that process or item a lot of thought and see if you can live without it. If so, find a place to put it and get it out of your reality. If in truth, you can do just fine without it, let it go. Don't spend another minute letting it clutter up your time or your energies.

Have you wanted to learn a new language or use a new approach to testing, but you are hamstrung by the practices you have to keep up and maintain? If there isn't a real compelling reason to keep them going, dare to set them aside for awhile. Examine their real utility. You may be surprised that all of the attention you've been paying to "essential reports, forms, charts, and processes" might actually be not that essential after all. Dare to do something different, and see how it goes. Consolidate processes where it makes sense, and dare to stop doing some things altogether if you are really just treading water doing them. The end of the year is a classic time to de-junk, de-stress and de-clutter. Take the time to do so, and you may find that your house, your desk, your work group, and quite possibly your sanity function way better by removing the thing that you think you have to keep around, but really don't. Give it a try, let me know how it works for you :).

Don't Rely on Your "Future Self"

For those who are familiar with my Personal Blog (mostly inactive now that I spend most of my writing time on TESTHEAD), I have a number of posts related to handling money and avoiding debt. I avoid debt for many reasons, but the biggest reason is that I do not trust the entity that I call my "Future Self".

My Future Self is implicit in any arrangement I make. Of course, I have only the best of intentions with My Future Self (I don't think any of us want to consider we will be slothful or shiftless or unreliable). Still, the truth is, My Future Self is just not as reliable and as pliant as I want him to be, and this gets more obvious the further out I look. Sorry for bringing this up again, but this point was really drilled home to me this summer. I had lots of plans that related to getting in shape so that I could consider making a shot at a "Legend's" division competitive run in the United States of America Snowboarding Association South Lake Tahoe Series once again (it has been several years since I competed, and I had high hopes I'd be able to get back on that horse again). Well, my bone breaks scuttled that plan thoroughly, certainly for this year. Imagine if I had bought a season pass and new gear with the hope I would be competing this winter! I would have made those purchases based on a Future Self that never materialized (and could not materialize).


When I take out debt (financial, technical, or other nature) I am, likewise, putting My Future Self on the hook to take care of it. In all cases, this involves interest (whether it's codified as an interest rate or not, any decision put off until later costs in extra money, extra effort, or extra emotions, bet on it). When I was younger, especially, I was much more willing to spend and pay later. I had the attitude of "So what, it's only money, I can always make more!" Well, yes and no. With small amounts, it's true, but did I really want to tie myself down to potentially a lot of work and effort later on for an experience or a memory that may not even be relevant those months, or years, later?


I have a back log of books that I have to read. Yes, have to, because I made a verbal agreement to review them. Many of them have been provided to me for free, and therefore, I have made a "gentlemen's agreement" to work through them. Thus, I have made sure that My Future Self must put aside the time to read, work, ponder and reason his way through them. If he does not, then he goes back on his word. That is something I do not want to see My Future Self (or My Current Self, frankly) ever have to do. No matter how I look at it, this is time I must spend on a verbal commitment. Of course, I believe this will be beneficial, a good investment both in the way of knowledge and experience, and I will have an additional artifact for others to look at and decide if they will consider it worth their time to read for themselves. Still, it's a process that depends on me carving out time to do it, and often, it feels like that time is harder and harder to come by.


In some ways, I think that "time debt" is the most dangerous of debts, even more so than monetary or technical debt (they are all intertwined, of course). This is, I'm sure, sounding ironic coming from me, someone who prides himself on being "frequently busy" engaged in many endeavors. It's one thing to be doing a lot of stuff. It's quite another to put ourselves into a time debt that we will never be able to effectively manage or eliminate. Additionally, there's no way to bank, stretch or ration time. You can't save up time to use on a rainy day; everyone gets the same 84,600 seconds; 1,400 minutes; and 24 hours as everyone else. Time can only be used, and any time we decide to do one thing, we are deciding not to do something else... and really, get the thought of "multi-tasking" out of your head. I do not believe humans can do it effectively. Even super fast computers don't actually multitask; they split their attention among a large number of processes and cycle through them. It's an illusion, and an expensive one when scaled out to a human life.


Consider this my way, both during this end of 2011 and beginning of 2012, to encourage everyone to "honor thy energy" (with thanks and pre-apologies to Merlin Mann ;) ), and please, do what you can to inconvenience and engage Your Present Self as much as possible, and don't overburden Your Future Self. They have enough working against them as it is without our help!

A TESTHEAD ST&QA Double Play :)

     In the October 2011 (Vol. 8 No. 5) version of ST&QA Magazine, I get to have a double play :).

     First, this issue is dedicated to the release of our book "How to Reduce The Cost of Software Testing" (available from both the Publisher, Taylor & Francis and from Amazon) and there are a number of stories that are focused on the topic. Many of them are all new material that was written after the book had wrapped. The cover story, coolly enough, is an excerpt of my chapter (Trading Money for Time: When Saving Money Doesn't (and When It Does).


Second, and really humbling is a piece called "What is the Cost of Testing?" in which Rich Hand interviews, well... me! We had a great conversation a while  back, and only about 50% of it appears in the article. Rather than structuring it like an interview, it's a narrative with many of my thoughts interjected. Maybe this is a better approach; Rich was able to edit out areas where I probably rambled a bit (LOL!). Still, I think it's an accurate and representative piece, and I'm happy to have had a chance to be a part of it.

So for those interested in getting a copy ST&QA, go to the SoftwareTestPro.com site and download it, or you can read both the Cover Story or my Interview right there on the site. If you do, please leave a comment or two, I'd love to know what you think :).

BOOK CLUB: How to Reduce the Cost of Software Testing (21/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry. Ths entry covers Appendix D, and this is the final entry in this series.
.

Appendix D: Cost of Starting up a Test Team by Anne-Marie Charret


For some organizations, it’s entirely possible that you don’t need a test team. You may have a culture of ownership of quality and your development team may be doing a very good job at being testers. It’s possible that your customer support team may be fulfilling the roles of testers (and I myself can state that many really excellent testers came out of the tech support ranks or spent significant time doing technical support; it helps train them to be customer focused and look at problems from their perspective).

However, what if that’s not enough? What if your development team and your technical support team aren’t able to handle all of the testing needed? What if you have decided you would like to see the quality and performance of your application increase? Are you sure that a creating a test team is the solution to your problem?

Some might want to see that processes are followed, that quality issues are addressed. That’s all good, but will adding a test team confirm that the company’s policies are followed? It might, but then again, it might just add another group that doesn’t communicate or use the process. Before testing can be called on to fix a problem, it’s really helpful to determine what the problem actually is.


A cost effective test team is one that meets your organization’s needs. Many times testers are brought in to solve a problem, but what they are addressing isn’t the real problem. Instead, they are addressing a symptom that points to a bigger issue. Why are there quality issues? What’s really the cause of them? Why do some companies need a test team and other seem to do well without them? Testing is not a simple commodity; it can’t just be “plugged in” and left to run. It’s strongly influenced by the culture and beliefs of a given company. Test teams taken from one company and dropped into another will not perform exactly the same (even with all members being the same people). The company itself shapes the test team to its value system over time.

Cem Kaner describes software testing as:

“An empirical technical investigation conducted to provide stakeholders with information about the quality of the product or service under test”


The term “stakeholder” means anyone who cares about the quality of the product. Stakeholders may be in any department (sales and marketing especially). Having these people or entities in mind as you test will inform the testing, what you do, and how you do it.


It’s possible you see the value of a test team, but don’t have the budget for one at the present. It still would be valuable to go through and see what you would need in the way of testing resources, and after investigating the potential benefits of incurring that expense (remember, testing doesn’t actually make money) you may decide that, down the road as the company grows, there is a benefit to developing and growing an internal test team.

“Why do you want a test team?"

A common reason why companies want a test team is they believe the tester will be the enforcer of software quality. That’s a common perception, and frankly, it’s a dangerous one. Michael Bolton, in his talk "Two futures of Software Testing" explains that:


“Although testers are called the quality gatekeepers…
• they don’t have control over the schedule
• they don’t have control over the budget
• they don’t have control over staffing
• they don’t have control over product scope
• they don’t have control over market conditions or contractual obligations


Testers cannot enforce quality, that’s not their mandate. Even if it is their official mandate, they still cannot practically follow through on it. A test team can *influence* overall quality to improve by doing the following:

• Testers can find bugs and inform developers
• Testers can identify risks that threaten the value of the product
• Testers can highlight visibility and structure issues within the team (they just can't fix them)
• Testers add to the overall knowledge of the system

Setting a realistic expectation for what the test team can and cannot do is essential to their success.


So you’ve decided to take the plunge and create a test team. What will its make-up be? Do you want an in house test team? Do you want an independent contract team that is off-site? How large do you want your test team to be? As I’ve said many times, my test team at my current company is dynamic. It has one dedicated resource (i.e. me) and at times we can call on others to help the process, often other people in our company in different roles, and especially our technical support people. As a dedicated and solo tester, I often sit with the developers and get to see what they see and understand a much as possible their environments and challenges so that I can help them meet their quality objectives.


Another approach is to contract with a company that has an external testing lab. They will be hired to do the testing and to report back on the status of a project. There are benefits to this. The external organization has the equipment, tools, and experience to handle a variety of testing challenges that an in house team might not have. They also can be used when they are needed, and when they are not, they are not part of your permanent payroll. The disadvantage is that they may not have as much familiarity with your organization and your expectations the way an embedded tester might. There is also the cost of time delays with turnaround of results, reporting results, and then following up based on the information provided. This can get to be significant if the external test team is half a world away.


One of the challenges any test team will face is that of the balance between manual and automated testing. A quote I’m fond of (paraphrased and not attributed, sorry) is that “the human mind is brilliant, articulate, inquisitive and slow. The computer is inherently stupid, lacking in any ability to think for itself, but it is very fast. Put together, the abilities of both are limitless”. Manual testing and automated testing (or my much preferred choice of words “computer aided testing”) need to go hand in hand. All manual testing may yield good results, but it may be too slow to be practical. Fully automated testing will be enormously expensive to implement, and taking the human out of the equation may cause you to miss more bugs than you would with manual testing. The point is, your test team will need both.


When does it make sense to build a test team? Ask yourself and the stakeholders the following question:

“Is my company willing to take the risk for shipping the product as is?”

A test team can identify risk, but they can’t prevent it. Developers who will need to fix code. Project Managers need to allocate time and resources to fix problems. All parties need to realize that testing is not a panacea; they can communicate about the state of a product or service, but it’s the development team that ultimately fixes it.

BOOK CLUB: How to Reduce the Cost of Software Testing (20/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry. Ths entry covers Appendix C.

Appendix C: Rapid Test Augmentation by Jon Bach

Jon starts this chapter with the analogy that we outsource things in our lives all the time. When we call a plumber or go to a restaurant for dinner, we are actively “outsourcing”. Why do we do it? There’s usually two big reasons (though there may be several others). First, we need someone to do a job that we can’t easily do. Second, we get someone to do something that we could do, but they can do it much faster. So why do so many have a visceral reaction when we hear the word “outsourcing” in testing? It’s because we often equate the word with “offshoring”; using a different country’s labor (usually overseas from the United States) to do work we either won’t or can’t do without great expense. Man, the arguments I’ve had over the years regarding that topic could take up several posts, and really, the question is “is the argument even fair?”. We have no idea if offshoring our outsourcing is really lowering costs and raising value… or do we? Jon seems to think he does. shall we find out :)?

Prior to working for eBay, Jon worked for QuarDev, which meant he was a manager for hire. Why would you call Jon? Probably because you need something done quickly, with a solid expertise, and a budget to allow you to do that in a way your current team couldn’t at that immediate time (not that they were not capable, but they just couldn’t do what was needed in that sphere at that time under the circumstances that necessitated hiring Jon, right?). One of the things Jon is famous for doing is called Rapid Testing (his brother James Bach gets the credit for inventing the approach, along with active help from tester like Michael Bolton and Cem Kaner. Rapid testing is the “skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny.” (James Bach, Satisfice)

Note, Jon’s team is not cheap. Some offshore serices can do “the job” for 5 times less. The real question though, and Jon is willing to stake his reputation on this, is that they can’t, not with the level of ability he an his team can do. Their philosophy is to make the sales conversation about value, not price.

As we’ve seen in this book, cost is entirely context based. One context doesn’t hold up to another. In this case, Jon is comparing labor costs and the fallacy that all testers are created equal, and there’s really little skill involved and one tester is synonymous with another (editors note; hang out with Jon for about an hour and test with him. I promise you will see the interchangeable testers fallacy is exactly that!).

An interesting development was taking place before the downturn in 2008 called “re-shoring”; projects that went overseas because time differences, language barriers, skills levels and other differences were significant enough that the cost alone was no longer the deciding factor. Many times, it made more sense for a value perspective to keep the work local. Sadly with the downturn, the flow seems to be going the opposite direction again.

To be a consultant means to go beyond the stereotypes and the bad jokes everyone has about consultants (jokes that tend to have a sting of truth to them). Jon’s strategy is to:

1. Charge a fair price.
2. Have some ideas about how to execute what you recommend.
3. Write reports that tell the story you need to tell.
4. Write simply.
5. Tell them what you found out and call them "findings."
6. Talk about what you hope to do next or how you see yourself being involved.
7. Discuss your code of ethics.

Questions to ask clients to help them get the best value out of the engagement:

1. "What's important to you?"
2. "What does success look like?"
3. "What's the worst that could happen?"
4. "What would you like to do less of?"
5. "What would you like to do more of?"
6. "What do you wish were different?"
7. "What problems are you trying to solve?"
8. "What's working?"
9. "What would you start doing, stop doing or keep doing?"

Jon likes to speak to the very top level of management, but also likes to spend time side-by-side with the testers to see how testing actually gets done. A lot is learned at both ends as ell as in between. Does the company value "heroism," and does that mean they they *rely* on heroes to solve major problems every time? Are there processes in place that prevent the heroes from becoming casualties? If so, are they actually applied? There’s a happy medium between heroism that’s excessive, and process that’s stifling.

Rather than polishing up a PowerPoint presentation, take out a sheet of paper during the pitch and ask: "Can we work right now on one of the problems you're having?" The way you probe for values - thoughtfulness, enthusiasm, and an earnest interest in the work – will help lead you to find the best fit for who you partner with.

Some Sample Questions FROM Clients 
Your bug database or mine?
Can I get the same tester as before?
How do you train?
Can I get resumes from your staff?
To what associations do you belong?
Can I see the templates you’ll use?
Can I customize the status reports you give?
If I need to postpone or cancel, what’s the penalty?
Can I talk to a tester in the lab?
What are your working hours?
Will you work overtime or weekends?
What’s your hiring process?
Will we be billed for the hours we don’t use?
How do you measure test coverage?
Why didn’t you catch that bug?

Some Sample Questions FOR Clients 
Your bug database or ours?
Can I talk directly to a developer?
What are your working hours?
Do you work overtime or weekends?
What’s your triage process?
When do you plan to ship?
Will we work onsite or here in our lab?
Do you use any existing tools that would be of help?
Can we see your existing bug database?
Did you want to devote time to regressions?
How often will you be giving us builds?
What are the minimum hardware requirements?
What kinds of users is this targeted for?
Has this been tested before?
Would you be a reference?

The “cost” of something isn’t just about price. It’s all of the context and values and weighting and considerations that go into the computation. Think of the average ink jet printer. They’re actually really inexpensive… the printer’s themselves, that is. The ink cartridges, though, can cost up to half the price of the printer!

List all of the things that matter to you – all of your hopes and ideals and values. Put them into a list and give a “gut-feeling” ranking to each of them on a scale of 1 to 10. Share it with the people on your team who are charged with making a decision. Add, delete, modify. But also stay alert to new context that emerges once the decision is made.

To know “the cost of testing”, you have to know the context of testing, the value of testing, and the questions of testing. You can buy a cheap car for $500 and maybe it’s just what you need. You can buy a new car for $50,000 and maybe it “pays for itself” in the benefits you get from it.

For the question of hiring a test lab to do rapid test augmentation, you may be able to reframe conversation to your stakeholders and decision-makers – helping them realize that you’re not hiring them to find bugs, you’re hiring them to assist you in gaining more visibility about the value you are offering to your customers.

BOOK CLUB: How to Reduce the Cost of Software Testing (19/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry. This entry covers Appendix B.

Appendix B - 25 Tips to Reduce Testing Cost Today by Catherine Powell

OK, wait a second... didn't I review a batch of "25 tips" yesterday? What gives?! Well, Appendix B is a strikingly similar set of 25 tips. In fact, they have exactly the same titles. The difference? Many of Matt's tips had to do with things that cut waste, but were organizational, and possibly out of the realm of the Individual Contributor Tester (i.e. the Tester doesn't have the authority to do many of those things). In other words, if I were an individual Tester, what could I do to lower costs where I don't have to ask anyone's permission to make an impact? That's where this "list of 25" comes in :). Look at them from the perspective of how any Tester can put these into play and make them effective. Also, don't try to do all of these things at one, do one at a time, practice it, master it, then move on to something else. Think of these like a pebble going into a jar. It may not seem like very much, but with enough pebbles, you'll ultimately fill the jar ;).

Tip 1: Cut your documentation to a minimum

Let's do what we can to make our documentation more DRY ("don't repeat yourself"). Have you been copying and pasting things into your docs? Stop that. "Incorporate by reference". Replace those copies with a simple note: "See XX", and then put it - just once - at that location.

Tip 2: Make the cost of changing the documentation cheap

Source control, wikis, whiteboards, they all allow us to share documentation and change it quickly and from multiple sources. Leverage that. Make internal reporting as quick as possible, and let the QA Manager or lead handle the more formal report (which should also be easier to do with this collaborative apprach). Start with your own team, then once you've shown the benefit by collaborating on the documentation within your team, share it with other teams and encourage them to follow suit.

Tip 3: Never be blocked

Make a wish list of the things you wish you could do but you don't have time for on your whiteboard, a sticky note, or in your Smartphone. Spend just ten minutes on this, you don't want a massive list. Then put the list aside. Next time you're blocked, pull out your list. Now's the time to do something on it!

Tip 4: Eliminate Multi-Project-ing

Individual Testers may not be able to choose if they work on multiple projects or not, but you can choose how you deal with working on multiple projects. Pick a single project and create a meeting in your calendar for you and that project - and spend that block of time working on that project only. The more often you can do this, the better. Minimize task switching. Turn off email, IM, phone, put a “do not disturb” sign on your chair while you are testing. Whatever it takes :).

Tip 5: Automate entirely redundant processes

Automation freaks some people out, because it sounds so huge and nebulous. Try some "computer aided testing" instead. Make a shell script or a batch file. Find a repetitive task that you've done at least twice in the last month, and create some "computer aided testing". Then check into the source repository so everyone can use it. Start small, get some ideas together, and build it out as you do more of it. Even a list of commonly used commands in a single file can be a huge boost.

Tip 6: Start with Soap Opera Tests

Soap opera tests are where you throw everything, including the kitchen sink, into your tests. Load them suckers up and let 'em run. You might find some spetcular issues :).

Tip 7: Start with quick attacks

Quick attacks are finding ways to do something hairy and over the top to inputs or outputs. Search for eveything. Load Hamlet into a text field (thank you, QAHatesYou, for my ultimate favorite Quick Attack ever :) ). If you feel fancy and want to go a little white box, then find every error message that a program is supposed to display, and force the program to show it. That's one of my all time favorite tests; lots of time is given to the happy path. Have they spent nearly as much time and effort on error handling? This is a great way to find out :). You get the idea.

Tip 8: Test Early

If you're in an agile environment, go through your last iteration and identify how long it took you to start testing each story after it was done, and come up with one idea to start each one faster. If you're in a more traditional environment, go find your friendly local developer and offer to spend half an hour working with him to "pre-test" his feature together. If the developer looks at you funny, offer to bring donuts (or beer, some people are just motivated differently ;) ).

Tip 9: Develop a regression-test cadence, and keep it light

Creating a "testing-oriented system diagram" so you can figure out what components relate to what other components. Use it to identify what needs to be regression tested and what you can more safely not run. Ask your team to expand on the drawing, discussing what fixes or feature changes might break other things. Have an architect validate your diagram. Again, if necessary, bribe with donuts or beer ;) ).

Tip 10: Test the things that actually yield bugs

Pick one area that is on the top of your "must regression test" list, and one area that is on your "we don't have to regression test" list. Then do it, with just one area and with just one release. Prove to yourself that it's okay to not regression test *everything* every time. Focus your testing efforts on areas that are at higher risk of having new problems.

Tip 11: Elaborate - and communicate - a test strategy or triage strategy

Create a status update that you can publish frequently. Publish it at least twice a week for the rest of the release. If your release is in under a month, publish the status update daily. Be sure to include the "must fix bugs" and a summary of new bugs found.

Tip 12: Decrease your time spent reproducing the problem

Next time you find an unreproducible problem, set a timer (an hour or two max). Now attempt to reproduce the problem and gather information. When time is up, send the unreproducible bug over to development with two or three ideas for extra logging or information that will help track down the problem if it happens again.

Tip 13: Do a low-cost beta program or pre-release

First, do a beta test of one. Get an account rep to bring a friendly customer to your offices. Let them use the newest software for an hour or so. Sit next to them and watch what they does and what questions they ask. Have a frank conversation afterward (thoughts, likes, dislikes, what's missing, etc.). Next, do a ride-along for half a day or a full day w/ an account rep to a friendly customers site, and shadow the customer(s) for half a day or a day. Keep quiet and take notes about how the customers actually uses the software. Share this information with your team. Learn from your actual end users. If the customer is open to it, record them with your phone, or at least take pictures. Pictures are worth a thousand words.

Tip 14: Recognize and eliminate project wankery

This might be hard, but give it a try anyway. Think of the meeting that you feel is the biggest waste of your time. Then make plans to skip it. Provide whatever informstion you need to satisfy those who feel you miust be there, but explain you can't make it for a pressing reason (no need to elaborate). The goal is to prove the necessary work *can* get done and that the product won't suffer if you don't sit in this meeting Swapping a meeting for a brief written report is a good bargain in service to your overall goal. You may not be able to do this very often or for many other meetings, but even if you can get out of just one of them on a regular basis, over time that can add up to a lot of quality testing time.

Tip 15: Stop fighting to manage the project

#1 Rule: NO WHINING. If you don't want to manage a project, don't complain about how it's being managed. It's probably not part of your job. Don't let the things other people choose to do interfere with the things that only you can do. Let it go.

Tip 16: If you're going to fight, win

Sometimes the decision really does matter! If it's truly important, you need to be able to sway the decision your way. Your reputation, your argument, and how much support you’ve garnered to your side will have a huge impact on the outcome. The next time you make a decision, sit down and write out why you're making that decision, along with the pros and cons of the choice you are making. Then give that explanation to your boss and ask for feedback. Make clear this is *your* decision, you're not asking your boss for permission. This lets you see if you argument is persuasive and if others will back you up on your decisions, and if you are really ready to "fight to win".

Tip 17: Walk around and listen

If you're a test lead or a test manager, take fifteen minutes and walk around the office in the morning. Repeat your 15 minute walking circuit that afternoon. Don't interrupt anyone, and don't talk with them. (It's okay to say "hi" back if someone greets you.) Just watch and listen, and notice who doesn't seem to be doing anything either time you walk by. After your circuit, sit down for 10 minutes with everyone who was not accomplishing anything and ask what you can do to unblock them or help them find a project to do.

If you're not a test lead or manager, see what you are doing with your time... no really, what you are really doing with your time? You may find that you are not nearly as focused as you think you are. Be brutally honest with yourself, and ask, OK, what can I eliminate? What can I minimize, and how can I get the necessary project time beefed up and finish what needs to get finished? Aim to decrease the distractions each day and boost the project stuff each day until you get to a realistic level of external tasks that have to be done, and as much project focus as you can reasonably perform. Seriously, nobody's going to be able to be 100% during an 8 hour day, but try to maximize where you can.

Tip 18: Write test automation that attacks the business-logic level

If you are using GUI tools and automating through the GUI, detemine how you might use another tool or rewrite your test to do a similar job from under the board. This might take some coordination with the development team but it certanly can be done. Determine what internal tools they use to get to the console or other hooks in the program, and then start working out tests that utilize them.

Tip 19: Develop a test automation library

This may be tougher for non coders in the test group, but think of the scripts and the automation you may already have. How much of it is duplicated in other tests? If you see a bunch of duplication, here's a chance to make a library or somehow centralize those items so that the can be called in a simpler manner. Personal experience: a neat trick in Cucumber is to take commands that get run a bunch of times and make "macros' of them by using the %Q|[cucumber line]| syntax. This lets you group various Cucumber commands into a one line call. DRY in action :).

Tip 20: Develop or hire expertise

Spend 15 minutes and figure out the expertise of each member of your team. Then take another 15 minutes to figure out where you need experts that you don't have. These are the areas you should be considering for training or hiring: to fill your expert gaps.

Tip 21: Get a return from conferences or don't go

Play "find an expert" the next time you go to a conference. Find someone with experience in an area your team lacks. Follow up with the expert - a quick email is fine - to establish a relationship. When you have a question about [fill in the blank expertise], you will be able to reach out to the expert. They don't have to be the famous testers. Often the ones that aren't are totally down with helping you and will often bend over backwards to help you get where you want to be. Note: don't abuse this relationship. They won't do your work for you, but if you do your homework and come to them with specific questions and showing you've already given your all, they may prove to be surprisingly helpful :).

Tip 22: Build a social network

Pick your favorite testing site (Software Testing Club, StackExchange, whatever) and answer one question. Next time you have a question, go to the same site and ask it. Do this regularly (maybe once a week), Over time, you will build a network of testers you can rely on. You'll also build a reputation and have public evidence of your own competence, which can be huge when it comes time to look for a new job :).

Tip 23: Examine invalid bugs carefully

Do some data mining in the defect tracker. Create two reports: (1) number of bugs marked invalid by reporter; and (2) number of bugs marked invalid by feature/module. Take that report and go talk to the reporter with the most invalid bugs. Pick a handful to walk through with him and spend 30 minutes brainstorming how to decrease that rate. Then take the report to the owner of the feature/module with the most invalid bugs. Pick a handful to walk through with him and spend 30 minutes brainstorming how to increase team-wide knowledge of his module/feature so the number of invalid bugs decreases [I'm not going to paraphrase this one, I think it's perfect as is :) ].

Tip 24: Build a test model that deal with the natural up and down staffing need for test resources

If you're already a tester on a team, spend 15 minutes brainstorming "what do you do when you're idle". A list of things testers can do when testing on the product is relatively light should be the result. Make a list of concrete actions you can perform. Next time you're idle, grab something from the list, and do it.

Tip 25: Ask the team to identify opportunities to drive out waste ... and implement them

Is there something in your processes that seems totally stupid and a complete waste of time? Kill it! Are there documentation steps that no one ever reads or does? Remove them. Tests that add little or no value? Weed them out. This is a gift that keeps on giving. As soon as you get good at this, you will find other areas you can trim away, too, and the better you get at it, the more areas you will discover. This is also a cycling list. As you cut wasteful activities, you will find room for more fruitful ones, and that fine, but remember, no fruitful idea will remain 100% fruitful forever. New processes develop waste of their own accord, and soon you'll be fining ways to "refactor" there, too.

BOOK CLUB: How to Reduce the Cost of Software Testing (18/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry.

Afterward

The editors of the book give us here a charge by way of a description of a game manual set up with a glossary. The point made is the last entry doesn't have a , it ends with Y, in fact, it ends with the word "You".

You: Have reached the end of this manual. We've taken you as far as we can; the rest is up to you.
Not get out there and play!

This is the official end of the book, except that it isn't. Over the next few days I'll be covering Appendix material that likewise deserve their own entries (see below), and of course, the conversation continues with the writers of the book. Most of us are very active on our own blogs, on Twitter and other mediums, such as the Software Testing or SW-IMPROVE discussion lists -- and on LinkedIn. If you try the ideas in this book, please let us know what you think. The important thing is that it's now time to put the ideas to work, so get out there and test!

Appendix A: Immediate Strategies to Reduce Test Cost by Matthew Heusser

So the book is finished, yet you may be thinking to yourself "OK, that's all great, but I need stuff I can do "right now" that can provide immediate payback. do you have any suggestions for that?"

Matt offers the following:

While testing less is an easy cost cut, it also introduces risk. The best way to help reduce the cost in an immediate way is to reduce or eliminate waste. The following strategies may help that, as well as provide additional value added benefits from trying a few things (25 of them, in fact):

1) Cut your documentation to a minimum
First you have to write it, then maintain it. Plus it only has value when people read it.
Instead of a comprehensive breakdown, focus on removing details that are not important to the testing effort. Keep those that would need to be known if you needed to hand it off to someone so you could leave town for a week. Is there enough there that the tester can read it and to the job? If so, aim for just enough documentation to do the job, but not so much that it becomes a time drag.

2) Make the cost of changing the documentation cheap
Perhaps use a shared doc, or on a network drive,  put it in a wiki, or hey, why not just keep it on a whiteboard?

3) Never be blocked
If a tester feels they are blocked or they can't do anything, they are generating waste. Instead, ask the following:

(a) Can I interview customers for test ideas
(b) Can I pair with the developers to learn the system
(c) Can I pair with another tester on a different piece of functionality in the same project

4) Eliminate Multi-Project-ing
Focus on a task at hand, and if you are blocked on it, see #3. Taking on side-projects and having to frequently context switch takes a lot of time. Focus on one area until you are done, then move on to something else if necessary.

5) Automate entirely redundant processes
While automation is sold as a panacea to all testing troubles, we know it's not. There's much more a real thinking brain can do at interesting points in the software that automation can never do. However, there's a lot of steps that are horribly repetitive and don't add any real value to the testing knowledge (set-up, navigation to key places, etc.). Absolutely automate these sets if you can.

6) Start with Soap Opera tests
Instead of testing for individual factors in a test, start out with testing the largest group possible, and if you see errors, whittle it down instead of testing each factor in isolation (which will take much longer).

7) Start with quick attacks
"Quick Attacks"are all about overwhelming the software with invalid input, too much input, and input that is out of range. If the software handles the obvious exception conditions well, it is likely in good shape. If the developers left holes and errors in the exception conditions, they likely left holes and exceptions in the main business logic, too. Quick attacks allows the tester to find bugs early, learn business rules, and perform a quick assessment of the software -- all at the same time!

8) Test Early
If you are use to working with software where an entire build is ship to you to test a new feature, see if there is a way you can get access to an environment where the feature is added and you can test it without having to wait for the entire build. The sooner you can test a feature in isolation, the sooner you can provide meaningful feedback the developer.

9) Develop a regression-test cadence, and keep it light
The days of shipping software once a year are disappearing fast. Most products are moving to incremental updates over the web, or much more frequent updates if its a web property entirely.

Regression testing, therefore, needs to be modified so that we don't have to run a years worth of test changes in a week,  or less. While there is a chance that any change could cause a problem anywhere else in the code, this is not universally true. By compartmentalizing tests that actually have a relation to other functionality, the regression tests can be made more efficient and focus on the areas where testing will actually yield results, which leads to...

10) Test the things that actually yield bugs
If you have regression tests you run for every release that never seem to find bugs, find a way to limit how many of those run. Maybe run them every third release, or one third of them per release, or just enough of them that would find a major defect if it were introduced in that code.

11) Elaborate - and communicate - a test strategy or triage strategy
List features by how critical they are, then go down the feature list in order for a "first pass". If you complete that "first pass", go in more depth. Make sure everyone knows how the decision was made, what the decision was - and the decision-makers feel like part of it.

12) Decrease your time spent reproducing the problem
If your team is spending a lot of time on bug reproduction, look for ways to lower it. Have the server store a log of every action. Find a tool to record exactly what the tester is doing and make a screen-cast that can be played back.

13) Do a low-cost beta program or pre-release
Can you segment your users and send some to a managed beta? If so, you could release early and engage the users in helping to test. You'll need some infrastructure (which users get which builds), and on web-based projects, some sort of production monitoring.

14) Recognize and eliminate project wankery
Getting better requirements, having more time to test, doing architecture and code reviews - those can all be good things. They can waste a lot of time, too. Experiment with these practices, but view them as experiments. After you've tried them once or twice (or if they are currently "mandated") ask if there will be less defects down the line, do we add value to the project, do we decrease overall cost by doing these things? If the answer is no, change the format, or drop them.

15) Stop fighting to manage the project
Instead of arguing over if the software is ready to release, make the defects visible to everyone and let senior management decide if it should be shipped. Imagine saying something like this:
"We're going to stop fighting you over issues a, b, and c. We yield to a, b, and c. We're going to focus on testing. If a, b, or c fail, don't complain to me. The decision is yours." Consider how liberating that might be.

16) If you're going to fight, win
If you do give on some issues, you may just want to pick a few battles. If you pick those battles, fight to win. Otherwise, you're just wasting time.

17) Walk around and listen
It's s good bet that there's a lot of other stuff going on that may look like work, but may just be busy-work leading to little value. How can you tell if that's happening? Walk around and listen. See what the testers are actually doing. They may think they are focusing on something important, or they may be "blocked" and not know how to proceed, but are too embarrassed to say anything. By doing this, you can then help coach or guide the testers to getting out of the rut, or determine what the blocker actually is. Is it their process? is it another person on the team? Are they just "socially loafing"? Don't wonder, find out :).


18) Write test automation that attacks the business-logic level
GUI testing is very expensive, and often GUI testing is really "change detection" testing, and can be very brittle if UI elements change. With setups and scaffolding and test hooks, testers could write tests that exercise business-logic level functions. These tests will general run much faster and be less brittle than GUI tests. Automated tests verify those dozens (hundreds? thousands?) of possible inputs for a given function. Test automation is an investment -- in the short run, testing costs always go up. Starting at the business-logic level might make it possible to see returns in days and weeks, not months or years.


19) Develop a test automation library
If you do want to test at the graphical level, or want to have more powerful business-logic tests, you may want to develop re-usable functions. For a GUI, that might be login (taking to parameters and pushing the login button), search, tag, etc. At the business-logic level these will probably be object-oriented functions.

20) Develop or hire expertise
As a test manager, you can foster expertise with brown-bags and pairing; when you look to expand your team, look for skills that would round out the same. Help develop your team so that there is redundant expertise where possible. Avoid the information silo.

21) Get a return from conferences or don't go
Expect employees to write two 'what I learned' documents: Things for us to do on Monday, and strategies to pursue over the next year. Use conferences to attract talent. Send your employees with business cards and a list of open positions. Use conferences to retain talent. Build the conference into each employees annual professional development goals, and he'll be more likely to stick around to attend it. Use conferences to grow your network of friends with specific expertise. If your staff comes back with ideas to implement on Monday, let them actually try them out!

22) Build a social network
Getting involved in local user's groups, attending conference, collaboration over the internet, all of these can provide an increasing list of people with skills you do not have, people willing, even eager to share ideas if you tap them on the shoulder. Instead of being "blocked" by a performance testing problem for a month and calling a consultant, a few questions to the social network, some calls, some questions, and a couple of days of exploration might solve the problem directly.

23) Examine invalid bugs carefully
How many bugs are being marked invalid, or 'wrong' for various reasons? Each of those means a tester and a developer both invested time for no benefit. If enough bugs are invalid, take a look at the reasons why and try to prevent it in the future.

24) Build a test model that deal with the natural up and down staffing need for test resources
Test projects have an ebb and flow. If testers remain constant, you'll either be stuck with too small a team or too big. Build a staffing model that allows you to scale up and down quickly. Perhaps bring in tech support to test, hold a beta program, bring in an outsourcer, or consider crowdsourcing like  uTest or volunteer a project to Weekend Testers.

And finally ...

25) Ask the team to identify opportunities to drive out waste ... and implement them.
A good tester is inquisitive, curious, and critical. The tester is likely to think "Why do we waste so much time doing X? I would think we could get by without it, or get the same benefit from doing Y."
So ask your testers what they would do to eliminate waste.

BOOK CLUB: How to Reduce the Cost of Software Testing (17/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry.

We are now into Section 3, which is sub-titled "How Do We Do It?". As you might guess, the book's topic mix makes a change yet again. We have defined the problem. We've discussed what we can do about it. Now let's get into the nuts and bolts of things we can do, here and now. This part covers Chapter 16.

Chapter 16: Rightsizing the Cost of Testing: Tips for Executives by Scott Barber

Scott makes the point early on that, unlike the other chapters in the “How Do We do It?” section, he’s not specifically writing to testers. He’s aiming for The CEO, CIO, or CFO and other senior execs that pull the trigger on decisions that they may really not understand. Even with that, the message here is still valid to front line testers. You may find yourself having a conversation with a senior exec, and you may be able to bend their ear and discuss this with them. Also, some day, you might be a high level executive, faced with the same issues. Wouldn’t it be cool to already know many of these details so you don’t make the same mistakes so many organizations do? Forewarned is forearmed.

For those who want to desperately believe otherwise, unless you sell testing services, there is no revenue in testing. There isn’t. It’s an operating expense like so many others, but it’s one that has an unfortunate detail going against it. Unlike some expenses that are quick to notice when they are changed, it may take several testing cycles to notice what happens if you cut testing from the budget. It may be months before the support calls increase, or before the cancellation notices start coming in, but in more circumstances than not, cutting testing is a case of “penny wise, pound foolish” and the result is usually a back tracking to real testing, and an even greater expense, until things normalize again. Testing has a disadvantage in the sense that, to most senior executives, unless they have spent time in the customer support area or in software development, testing is completely of their radar. We often joke that there are two experiences that come with being a tester. Either you are grudgingly appreciated because you find issues, or you are ignored when you don’t (even when the code is well developed and there’s little to really make headlines discovering. We are not celebrated when things are going good. We’re really only visible when things aren’t. Not to mention we’re often seen as the bearer of bad news, and really, who wants to hear that?

For me personally, I’ve been lucky in that many of the companies I have worked have been small enough where I had easy access to senior executives. I’ve also worked with companies that were significantly larger and where they indeed didn’t seem to have any idea what their testing organization was doing, and made decisions that seemed logical at the time but came back to bite them later on. If you are the executive reading this, perhaps a change in approach can help.

Executive Tip #1: Change the Question

Rather than asking “What value are we getting for what we are spending on testing?” instead ask “What value do we want to achieve through testing and how much are we willing to invest to see it actually happen??” Trying to calculate the ROI on testing is a lot like trying to figure out if paying for auto insurance is worth it. It may not be for years and years, but al it takes is one big wreck without insurance and the numbers change immediately! Instead of waiting for a “testing actuarial table” to appear, identify the areas you want to see your business increase value related to testing, prioritizing the areas and then determine an acceptable cost to meet those goals.

Executive Tip #2: Focus on Value to the Business

Most of the time, testing has been defined with the value to the project, rather than the value to the entire organization. How do we do that? We help the company pass regulatory audits, if they are relevant. We can provide defense against clams of negligence, faulty or bad advertising, or perceived breaks in service agreements. In short, good testing can help us not get sued (never a guarantee, but the likelihood of getting sued goes way down with vigorous testing). Testing can help the support staff deal with issues quickly and effectively. Testers can help provide stability between releases. Testing can help identify risk and areas where rick can be mitigated. Testers also work well as product trainers, as they tend to know the product better than just about anyone not on the customer support desk (and yes, that often means developers, too) Testers find discrepancies between requirements and what is actually being delivered. Testers detect issues earlier in the product lifecycle (especially if they are discovered before customers see them). Testers free up time for developers to focus on new feature development. Testers act as the advocate for the end user regarding usability issues and prioritizing issues from a customer’s perspective. The likelihood is that, seen in this light, there should be plenty of value identified in testing efforts.

Executive Tip #3: Distribute Testing Costs Carefully

I’ll confess I’m not an accountant, so much of this is outside of my area of expertise to comment on. Suffice it to say that, by spreading the costs of testing around among an organizations divisions and having the divisions be financially held accountable for the areas that they are responsible for, then testing costs can be placed in the areas that make the most sense for the organization and not impact other cost centers needlessly, and at the same time, each area can provide the support and financial backing necessary for testing to be effective across all divisions.

Executive Tip #4: Demand Accountability from Managers of Testing Programs

Executives have a need for information that they can quantify. Testing is often difficult to fit into that paradigm, but it can be done. Executives need to make clear what information they need, and why they need it. Testers need to make clear what the information they can provide actually means. Instead of asking for a particular measure or metric, executives that understand the value that testing can provide will focus on talking about, and trying to determine what value they are hoping to receive and by collaborating with testers and test managers, they can develop a metric or measurement that accurately represents that value. From there, the managers of the testing program (whoever they may be) need to know that they will be held accountable to that mutually determined metric, and that the testers know that they will need to modify their approach to ensure they can provide that metric.

Executive Tip #5: Keep Tactics at a Tactical Level

We’re all aware of the fact that we are often expected to do more with less, and how well we do more with that less determines if we get promoted or if we get sent packing. Scott points out that in his experience, more times than not, test programs are often cut by people who have little or no involvement or understanding of what the test team actually does. The only ones who can really determine what costs can be reduced without significantly degrading the value of the testing program is, well, the test team! How can they help this process? Well, the rest of the chapters in this book might certainly be a good place to start :).