Strive to Show Evidence of Skill: 99 Ways Workshop #35

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.

My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions.

Suggestion #35: Read the ISTQB syllabus from start to end - then use it as a map of the box you need to be thinking outside of! - Geekonomicon

Some people are sure they know where this is going to go. I'm hoping I'll surprise you ;).

Just like any question that's related to BBST may be taken with a grain of salt by some because I am directly involved in its focus, direction and delivery, so I should expect that some will take anything I have to say about ISTQB with a grain of salt as well. Isn't there a conflict of interest for me? If I'm negative, won't that be seen as "of course, he stands to benefit of BBST is seen as a better model?"

I'm going to do my best to avoid all that… not because I don't want to have the debate, but because I want to address what I feel is the bigger picture.

Workshop #35: Regardless of affiliation or ideology, aim for evidence-based proof of your skills.

The BBST course materials can be found here . If you have not downloaded or read the ISTQB syllabus, it's here. While I'm at it, take any testing course, book, video, or resource that you want to put to this test. Take my entire blog and everything I have ever suggested as a good or helpful idea. I'm willing to expose it to the same scrutiny.

Read through all of them.

Make sure you understand what they are saying. If you are not sure what they are saying, highlight sections and review with other resources. Compare what sources have to say. Note contradictions or differences in wording.

Do you agree with the elements as presented? If so, apply them. If not, refute with actual practice or data why you disagree with them. All principles are open to being tried and debated. There are no sacred cows.

Regardless of what you feel you know or understand of any of the areas and topics, ask yourself "can I apply this, and does applying this help solve a problem, or help me identify a genuine issue?"

As you try each area, see how long it takes, see the quality of information you collect, and see if the practice makes sense for what you are doing. If it does, use it. If there's a better approach, use that. If an approach doesn't make sense for what you are doing, don't use it.

Bottom Line:

Regardless of the value or lack of value of whatever approach, school, discipline, association, etc. at the end of the day only one thing matters, and that is demonstrable skills. No certificate of completion or claim of accomplishment matters if, at the end of the process, you are unable to demonstrate actual skills and proficiency in the area of testing you need to. 

Let me put it this way… I care very little about what paper you hold at the end of the day. I care a great deal about the skills you can bring to testing problems and helping to find issues and represent customers and stakeholders. Strive to show genuine skill, and be willing and enthusiastic to show that skill. Every thing else is irrelevant.

Guru99: Aiming to be the NetTuts of Software Testing?

One of the more interesting aspects of working on this site over the past few years is the fact that people reach out to me to ask them to take a look at what they are doing and to comment on it. I’m honored to get these requests, but lament the fact that I don’t have enough time to really get in and do sites like this justice. I decided to change that and take a look at a site recommended to me called Guru99.

Guru99 bills itself as a site that aims to help testers learn concepts and tools by actually practicing what they learn. They create tutorials for a variety of topics, some of which have video clips that the user can watch and review.


The topics that are covered range from commercial tools such as QTP, Quality Center, Load Runner, etc, but also covers topics that are open source such as Selenium, MySQL, and also looks to cover software testing as a general discipline. In addition to its own hosted video links and comments/commentary, there are also links to other tools and sites to be able to learn the details needed. For those familiar with NetTuts, I think Guru99 is trying to be a software testing analog.


Some of the material is definitely geared towards those who are aiming to cover the certification space. Many of the tutorials are worded and geared towards what has become standardized in the industry over the last few years. I bristled at some of the phraseology (software testing is an activity to check whether the actual results match the expected results and to ensure that the software system is defect free), and while I might reword the text and presentation a bit (there’s no such thing as a defect free product, and software testing does not make a product defect free; it does give us the opportunity to determine which issues are the most critical and that deserve the most focus), the overall message is still valid.


Each of the videos provided range from two to ten minutes, so they are following the Khan Academy model of short, focused discussions, where the user can play them multiple times and review them as they consider the details. This is much preferred compared to trying to go through longer videos, etc. It works much better with my own shorter attention span ;).


If I had to offer a direct criticism, it would be that a number of the videos use a digitized speaking voice to cover the more general topics, while more specific topics and sections are covered by actual presenters. For me, personally, the computerized voice is a distraction. While it’s meant to add clarity and detail, the problem is that much of it comes across as flat and challenging to listen to. I much prefer individual presentations, even if the presenter has an accent. No matter how well tuned the digital voices are, it’s much more engaging to have an individual attached to the topic be the one presenting the material. This way, the vocal nuance can be heard and followed. there’s also no way to insert passion into a computer generated voice (well, there is, but that would go well beyond what the tools and tutorials are meant to do).


Some may look at the materials and say “this is so old school”. This is traditional software testing. They would be right. This site covers several of the traditional tool products of software testing, and they would also be right. The material seems to be geared towards those looking to get certified. Again, true. There is some coverage of new ideas and open source materials, and again, yes, you would be right. The material isn’t cutting edge en masse, but it is a good representation of testing as it currently exists. For those who want to get into software testing, and are looking to work at a Fortune 500 company or at larger corporations, much of the material presented is very much on point. Long time testers may feel that the site is missing some key details, but as an overview, I think it covers the primary topics adequately.


Would I like to see more about context-driven testing, or other topics that are not currently represented? Sure. The cool thing is that they encourage others to help them do that. As of right now, there are three posted externally developed articles/tutorials available on the site, and I suspect they would publish more if people were to be willing to post them (hint hint ;) ).


Bottom Line: For experienced testers, there may be some interesting nuggets here, and some things that you might find yourself saying “hmmm, i didn’t know that!” For first time testers, there is a lot of information here, and a lot of it is quite good and direct, especially for working in traditional environments. For those looking for specific tool knowledge of the listed tools, there’s a lot of good information on these tools presented in a way that might be very helpful to those in that context. For those looking to add information about the state of testing, have it be readily available, and get into the hands of those who might most benefit from it, this looks like a good place to go.

TESTHEAD REDUX: Aikido and the Role of Certification

When I wrote the original post for this back in 2010, I think it was the first time I was willing to break out and say "I don't even know what I'm hoping to find with this". It was prompted by my looking at a variety of "certification" options out in the testing market at the time. Most of them I had just started to hear about, many of them were somewhat nebulous, all of them made me feel somewhat uneasy. At the time I said the following (note, I used my experiences with the martial art Aikido as an analog to my understanding of the certification landscape at the time):

In my mind, this is the big thing that is missing from most of the certification paths that I have seen to date. There is a lot of emphasis on passing a multiple choice test, but little emphasis on solving real world problems or proving that you are actually able to do the work listed on the exam, or that you genuinely possess the skills required to test effectively.  

The other issue that I have with this is that, just like in an actual real world confrontation, some of the best practitioners of Aikido may not be the best at articulating each and every step, but my goodness they are whirlwinds on the mat and on the street! This is because they are instinctive, and their training has been less on the intellectual explanation and more on the raw “doing”! 

The reason I mention these details is that I still, all these years later, have yet to find a true certification that actually leads to the goals I am after and desire, but I also have found several examples of exactly what I want to see certification become. In short, I want to see a certification that really lives up to the principles of Aikido. I want to see testing as a martial art in its own right (with perhaps a de-emphasizing of the "martial" aspect. Perhaps a better phrase would be a "philosophical art").

Before I get too far into this, I will say already that the three things I am going to suggest need to be taken with a very large grain of salt. Why? Because I have a vested interest in all of them, but not for the reasons you may be thinking. A disclaimer... I make no money from any of these endeavors. In fact, in some ways, I forego earning money in other ways so that I can champion them. If I wanted to make myself the equivalent of the impoverished warrior monk, or a Zatoichi, I may have found the perfect recipe in these three examples ;). Nevertheless, I do them because of the value that I believe they provide, and from the anecdotal value that others come back to me and say that they offer.

BBST - BBST is the Black Box Software Testing courses offered by the Association for Software Testing and others. Note, you can get very close to 100% of the benefits of BBST without ever taking a class. All of the materials (the lectures, the course notes, the readings, etc) are available online. What's not readily available online is the course quizzes and exams, and the ability to be coached by other testers who help instruct the course. There is a cost associated with it ($125 for the Foundations course, $200 for the Bug Advocacy and Test Design courses), but the costs are used to pay for the hosting of the instances of the class, the servers, and administrative overhead. At this point in time, every Instructor for BBST is a volunteer, i.e. we don't get paid to do what we do. 

Weekend Testing - while BBST is one of the best direct trainings out there, Weekend Testing is, IMO, one of the best organized skills workshops held on a regular basis for testers to sharpen their swords on a variety of topics. Weekend Testing works on a variety of levels. It has much to offer the beginner who wants to learn how to test. It has much to offer the intermediate tester who wants to mentor newer testers, and likewise learn more themselves. It has much to offer advanced testers who can work to develop their skills as leaders by facilitating sessions and designing interesting and unique content to talk about, learn from, and make for a positive influence in the broader testing community. Also, unique to Weekend Testing is the fact that every session is archived. If you would like to show someone just how much you contributed, it's there in black and white, for all the world to see.

Weekend Testing has several chapters that are in various stages of operation. Chapters have been formed in India, Australia/New Zealand, Europe and the Americas. Currently, the India and the Americas chapters are the most active, but all it takes is willing folk to facilitate and have a strong desire to help testers improve their craft for more chapters to open up where they are needed (hint, I would have no problem seeing a South America Chapter develop, and we have yet to see a chapter develop in Africa. So there's plenty of room to grow, as far as I can see :) ).

The Miagi-do School of Software Testing - this is the one that probably means the most to me, and yet it's the one that will, guaranteed, never make me rich. Well, none of them will, but unlike BBST and Weekend Testing, which could be used as a marketable option or product, Miagi-do cannot. Actually, I should say it will not, as long as the founders have anything to say about it. It's not a not-for-profit. It's a ZERO-profit. It's also a ZERO-income enterprise. No money ever changes hands, and likely, no money ever will. People have to seek out the school, have to show they are willing, have to face multiple testing challenges, and actually put in a lot of work that leads to the betterment at large of the software testing community. 

Everyone's path is different, but my path was through signing up with AST and learning about the BBST classes, taking them as a participant, and then offering to teach them over the past three years. It included my producing a podcast dedicated to software testing topics, and frequently researching and presenting my own findings in episodes where I was features as a guest or a panelist. It involved my getting into Weekend Testing as then offered in Europe, and making enough of a commitment to it to be considered knowledgeable enough to facilitate, and then bring Weekend Testing to the Americas, where I have fostered its growth and development (along with several others, to be fair) for the past two and a half years. It involved writing in many areas, including the book "How to Reduce the Cost of Software Testing", as well as multiple articles for other distribution channels (ST&QA, Techwell, Tea Time for Testers, The Testing Planet, plus guest blogs for numerous companies and outlets). 

In between all of this, I have sat for and failed, and then later passed, several testing challenges, each one with the idea that I would demonstrate to my testing peers what I knew how to do, and what I didn't know. The Black Belt/Instructor level that I hold in Miagi-do may be laughed at by some. What does it mean, really? On paper, and in the eyes of HR departments, probably less than nothing. If, however, you and others out there feel that the talks I have given, the sessions I have facilitated, the courses I have taught, the articles I have written, and the podcasts I have recorded have helped, in some small way, to the improvement and betterment of the software testing community, then my black belt speaks volumes. In fact, it's my hope that everything else I have done, and will continue to do, makes the mention of a black belt completely irrelevant. 

In short, I am not my "certification". I am the ideas and the experiences that went into it. The fact that my certification is one that I have made for myself, surrounded with like minded people that I respect and admire, means more to me than any certification that can be given to me by any "officially sanctioning body" and yes, I'll include my Bachelors Degree in that list of "inferior certifications".

While a "certification" may carry someone into a second round interview, I will much frankly prefer to see my fellow Miag-do ka, BBST instructors and Weekend Testing facilitators on any project I would hope to lead and own. Why? Because I already know what they can do. I've seen it multiple times. In a dark alley situation,  I already know they can fight, and I also know they won't run away :)!

Forging Our Own Future

A bunch of people that I follow on Twitter are attending Agile Testing Days, which is taking place, well, right now, in Berlin. A great meme started getting posted and a lot of people jumped on it, including me. I think it’s worth repeating, and yes, I’m gonna’ pontificate on it a little, too :).


Ready? Here it is:

We are a community of professionals.
We are dedicated to our own continuing education and take responsibility for our own careers.
We support advancing in learning and advancing our craft.
We certify ourselves.


I love this! Seriously, this should be the manifesto of every tester. I want this on a T-shirt! I’d wear it proudly.


Now I’m sure some of you are saying “come on, what’s so amazing about this?” What’s amazing is that there’s a groundswell of testers that are actively pursuing making themselves and their communities better. We aren’t waiting for others to recognize our worth, we’re willing and able to put it forward ourselves. We are not waiting for others to decide what we will learn and when, we are getting together, formally or informally, to improve our craft at regular intervals. We are developing skills that go beyond testing, and that will help us develop our craft well into the next couple of decades, and we are not asking permission.


I appreciate the last statement most of all; we are willing to put our own actions, words, deeds and efforts into what defines us, and not relying on a piece of paper to tell others that we are “certified testers”. I used to be ambivalent about “Certifications”, since at least in my neck of the woods, they don’t seem to mean anything (at least not to any employers I’ve ever interacted with). But really, that’s not the point. I don’t want a piece of paper saying whether or not I’m good at what I do. I want my reputation to say whether or not I’m good at what I do.


Seth Godin inspired in me a desire to try something audacious… my next job, wherever it will be, will come without me sending a resume. Say what?! Seriously, that’s my plan. No, I’m not currently looking for another job right now, but should that time come, I want to have it happen because:

a. I’ve been sought out by people who have heard of me.
b. I have a list of highly respected references that will blow their minds.
c. I will have product that they can touch, feel, and work with that will make them go “Oooh!” and “Ahhh!”
d. I will have a blog so compelling that they will have to sit up and pay attention.

…plus other things I haven’t yet thought of.


Now, before you all think I’ve bought a first class ticket on the “full of himself” train, no, I absolutely do not think I am there yet. I’m probably light years away from that right now, but in a very real sense, that’s where I want to be, and that’s what I want to work to achieve.


Godin’s comment invariably comes with the answer of “yeah, that would be great, but I don’t have any of those things, to which he follow up with “yeah… that’s my point!” Meaning, if we don’t have those things to point to, what makes us think that we are that indispensable person, that go-to person that we hope and strive to be?


The key take-away with this current meme is that we and our community will foster the future, if we make the decision to make that future happen. It will not happen just by our wishing for it, it will not happen just by us waiting for it. To quote a favorite radio host, we have to make the decision to “get up, leave the cave, kill something and drag it back home”.


Vivid impression? I sure hope so!

Wednesday Book Review: Software Testing: Fundamental Principles and Essential Knowledge

Through the years, I have come across a number of books that I have used and valued. These may be new books, or they may be older ones. Each Wednesday, I will review a book that I personally feel would be worthwhile to testers.


If you were to talk to someone who was going to go into the testing field, and they said “I want to get something that would really help me get to the heart of what I need to know quickly and efficiently”. Given that, what would you recommend? I think the book “Software Testing: Fundamental Principles and Essential Knowledge” by James McCaffery would be a good first start.


First, this book is very thin. I don’t say that to criticize, but to extol a virtue. This book is focused clearly and distinctly on what the author has determined to be, in his opinion, the fundamental and essential. This book is terse, and focuses on a number of concepts directly. Each concept builds on the previous ones, and each chapter gets the user from basic understanding into actual testing scenarios, with a review question and exercise at the end of each section. The answers to these review sections are given in the back of the book. 105 pages is all there is from the Introduction page to the last review question answer. So what get’s covered in 105 pages?


Chapter 1: Software Testing and Test Cases. This section runs the tester through a whirlwind tour of test cases and test suites, and how test cases are put together and what they are meant to cover.

Chapter 2: Fundamental Mathematical Techniques. Combinatorics, Statistics, and some details about cryptography are covered, as well as a section on Pair-Wise testing (which made me laugh a little considering my blog post of yesterday :) ).

Chapter 3: Project Management Concepts. A quick tour through the Software Development Life Cycle, Test Documentation, writing of bug reports, and reproducing issues dominates this section.


Chapter 4: Core Principles. Various methods of software testing are discussed, including manual vs. automated, black/white box testing, performance, stress, configuration and localization testing.


Chapter 5: Essential Knowledge. This is a collection of miscellaneous test areas, including Character encoding, XML, databases, HTTP protocol, and networking testing and troubleshooting.


Chapter 6: Units, Modules and Components. This section talks about details associated with Dynamic link libraries, module testing, some details about unit testing, the Windows System Registry (noticing a bias here? Yep, this book is heavy on Windows testing), and creating stub and mock tests.


Answers to Review Questions. A section at the back of the book gives the answers to the review questions in each of the sections.

No question, this book takes the approach of grabbing the user and diving in. There’s little in the way of a gentle curve or illustration for points. The details here are focused and meant to help the reader get right into the thick of things. The good news is that the book is not designed to have to be read through from front to back. Any of the areas can be read in any order, and each has a review question at the end of each section to check to see how well you have retained the ideas.

Disclaimer: this book is associated with a certification program offered by Volt Information Services, Inc. and as such, the questions are written as though they are part of a practice test for a test certification. Some of the answers to the questions did have me scratching my head in a few places (not that the answers were wrong, just that they seemed to be strange choices based on the material presented).

Bottom Line:
If you are a beginner tester, or if you want to learn more about basic principles of software testing, this would be a good first book, but I wouldn’t recommend this as the only book a tester get. While it covers many areas, there are lots of other areas that get little to no coverage. Supplementing this book with the test fundamentals found in Kaner, Falk and Nguyen’s “Testing Computer Software” would give a beginning tester a great foundation to build on.

Aikido and the Goals of Certification

I think it is only fair to state that, at the current time, I do not hold any tester certifications. Therefore, please realize that what I am about to say may ruffle feathers, and some may have counter arguments to what I have to say, and there will be those who will say that I don’t know of what I speak because, hey, I don’t have a certification, so how would I know how valuable they are? The short answer is this; I have nothing but my own observations and my own experiences to speak to this, but I will offer that I am perfectly willing to admit that I don’t know what I don’t know.

Having said that, I will give a simple comparison to a sport I participated in for awhile. Back what seems like a lifetime ago, I participated with a dojo and practiced the sport of Aikido. I enjoyed it for a time and learned a lot about the techniques necessary for handling falls, disarming opponents, and performing and receiving throws. The interesting thing about Aikido is that you really cannot progress unless you show your ability, and there is a group review of your actual skill, in real time, under adverse conditions (the higher you place, the more difficult the tests become). In short to move out of the ranks of the “Kyu” (the student ranks)  and become a “Dan” (the master ranks) you have to really show that you know your stuff and prove it over and over again.

This brings me back to my current impression of certifications (and notice the words I chose to use, this is my opinion colored by my own experiences or lack thereof)… most of the Q.A. certifications that are mentioned and are bandied about focus on taking a multiple choice test. If you pass the multiple choice test, you are certified. In my view, this is like going into the middle of a dojo and testing for a rank by answering questions. While it may be great to have the book knowledge as to how to disarm an opponent with a knife and pin them, would you feel confident in this student’s abilities if all they did was answer a question about how they would disarm someone? No. Would you feel confident if you physically witnessed them disarm several opponents armed with knives? Yes!

In my mind, this is the big thing that is missing from most of the certification paths that I have seen to date. There is a lot of emphasis on passing a multiple choice test, but little emphasis on solving real world problems or proving that you are actually able to do the work listed on the exam, or that you genuinely possess the skills required to test effectively.  The other issue that I have with this is that, just like in an actual real world confrontation, some of the best practitioners of Aikido may not be the best at articulating each and every step, but my goodness they are whirlwinds on the mat and on the street! This is because they are instinctive, and their training has been less on the intellectual explanation and more on the raw “doing”! Don’t get me wrong, both skills are important, especially if one of your goals is to teach, but if I had to choose over the person who could analytically explain the situation but didn’t have the physical speed or dexterity to handle a situation quickly and effectively, versus the person less apt to explain but can quickly overcome an opponent, give me the latter any day. Likewise, in the certification sphere, I think it’s safe to say there are a number of testers out there (maybe a few of us have had experiences with them) who can pass a test, but in the real work crunch situations, may not perform as well as the paper says they can.

Now, again, it’s not my intention to poke at certifications, especially since I’ve already admitted that I don’t currently hold any (and for that matter, I only went through a couple of Kyu ranks in Aikido before life and its myriad of distractions took me away from my studies, so feel free to take that for what it’s worth, too), but I have watched a number of the higher ranking exams and witnessed the awarding of a Dan rank, and I can say with a surety that that was a person I would love to have with me in a rough neighborhood. I saw when she walked into a dojo that she had the appropriate trappings, including the black Hakama and belt that officially showed she was a Dan level Aiki (what we stateside would call a “black belt”), but she more than demonstrated that she deserved that Dan ranking by her movements on the mat.

Likewise, I am looking for a similar way to show my skills, or to develop them (as I say in the sub-text of this blog, this has to do with the mis-education and re-education of a software tester and others who may be dealing with similar challenges).  Were there to be a national or international and officially recognized certification that truly did such a thing, I would be greatly receptive to learning and sitting for such an examination. It’s very likely that I would learn a lot about what I did not know, and that, metaphorically speaking, I might be able to explain some situations but not fend them off in real practice, or be able to fend well but explain poorly. 

Kinda’ like real life.