Wednesday, November 13, 2019

Getting Practical with Practitest: A Broad Sweep at a Broad Product

Before I get started, a full disclosure. I was asked by friends at PractiTest if I would be willing to have a look at their test management software and talk about it on TESTHEAD. To that effect, I said yes. Beyond that, they have no influence on what I say and post here. I have given them permission to use my comments as they choose to so long as they reference this original article here.
And with that out of the way... :).
I must confess, I have done fewer product reviews with my blog over the years than I had thought I might do. Part of it comes down to a fundamental issue; what I use a product for may be very different than how others might use it. Still, I think it's safe to say that many of us have some similar goals and attitudes about the tools we use and how we might take advantage of them. On the whole, I am a fan of products that do their best to help guide the user with their use, do not bury arcane but useful features under layers of difficult to arrive-to processes and reward a user with regular and frequent interaction. Thus, it is with that idea and approach that I' going to look at what can be either one of the most helpful or most frustrating areas of software testing, that of test case management.

I have worked for a variety of organizations over the years and the methods of test case management have varied radically. Some places have literally only used index cards that get shuffled around (no joke, there were no permanent electronically documented procedures, or very few if I was being totally honest. Great from a free and exploring point of view, terrible in that cards just plain got misplaced and lost (and yes, that happened more times than I want to admit, best intentions notwithstanding). 
On the opposite end, I have also worked with very top-heavy documentation systems that were a chore to use and so burdensome that it felt like most of my testing time was spent writing up test cases and leaving me precious little time to actually test and explore systems. Wouldn't it be nice to have something in between? Something that allows for a less rigid and formalized structure if you don't need it but has the flexibility and power of documenting and traceability if you do? 
Practitest aims to do that. I aim to see just how well they pull that off ;).



What is Practitest?


First off, Practitest is an online, SaaS product offering. You access the product via the web and you interact with it via web browser. However, it may be easier to think of Practitest as an onion with layers of functionality (cue Shrek jokes, in 3, 2, 1...). However, that layer metaphor is quite useful,m in my opinion. By looking at each level and layer, it's fairly easy to get from one part to the other and actually have a handle on how everything interrelates.



The system is structured in several key areas. You have the ability to define your test requirements, store and keep tests that you develop, compile and follow test runs, track issues, and create reports, as well as customize and integrate with other products if desired. For today, let's just focus on the core product and getting productive with it.

Our Sample Test Flow
Let's take a simple example of logging into the system...

side tangent, have you ever noticed that most tutorials tend to focus on a log in screen? This stands to reason because it is often the first thing anyone does with a product unless there isn't a login required. Does it make sense to start there? Maybe not but it is easier to describe rather than walk through an entire installation script or break down every component of a search test.

In this case, there are a few interesting wrinkles that logging in with the product I am testing. We have a standalone login, an LDAP login and we also use a single sign-on built around iPaaS. each of those deals with different paths in the code and in certain scenarios takes us different places.

Step one, let's create a requirement(s). it can be as simple or as complex as we want. In my case, I want to just define what I hope to do and how I intend to accomplish that task (in this case, three login types):

- standalone

-LDAP

- iPaaS

Each of these we would be expected to create a separate test case but one requirement would suffice. Fortunately, the requirements section is the first thing that we interact with and adding information is quick and direct.


By adding requirements, we can capture in a little or a lot of detail what we want to cover. from there, we can go to test cases and define what we want to test. Practitest has two modules that allow for scripted or exploratory testing. I want to say that I appreciate the fact that Exploratory tests are built-in as an option because in many environments, there are questions as to what Exploratory testing is or means and a bias occurs because they are not as readily documented. Practitest handles that nicely. Also, if you need to get into the details, you can do that as well with the scripted tab.




Additionally, if there is a need for procedural and specific test cases to be recorded, those can also be listed in the scripted section.





A nice thing that is also included is the ability to include test cases that have already been written. Do you have steps you find yourself doing over and over again? Don't write them out again and again, include them where needed. this also is helpful as you start to consider which tests can be automated. as you get the hang of writing things out, you notice where you can simplify or determine repetitive steps. saving those as what I refer to as "test snippets", you can call them up and use them as often as needed.

Connecting the Dots
As I work my way across the toolbar, I see that I can trigger areas and examine what I am testing and writing down as I move from left to right. This flow is straight-forward and doesn't bog the user down in a lot of details or needless repetition. If I have to highlight a great strength of PractiTest, it's that it doesn't force me to be repetitive needlessly. Additionally, it allows test sets and test runs to be as specific or as general as desired. is the testing exploratory? Set a timer and go. capture and post what you find. Is the test a procedural one? Great. Run it and then see how long it takes to do it (the timer runs automatically in the background). When all is said and done, you can get a compilation of test runs, what was accomplished, what was recorded and what requirements those tests are associated with. Found issues, log them in the dedicated issue tracker, track them back to test, which trackback to requirements (again, right to left on the board). If I was a test manager, I would find this to be easy to understand, easy to track and it wouldn't take me forever to figure out what was going on.

Overall Thoughts

An impression I keep getting as I use Practitest is that the company values testers' time and appreciates that the best thing they can do as a tool is to provide the feedback necessary when necessary and get out of the way when it doesn't need to be there. Short of being specifically macro'd into a product where a right-click could allow a user to track what they are doing in real-time within an application, Practitest does a nice job of staying out of my way and I mean that as a compliment. Granted, this is the first blush and there's a lot of stuff I could talk about (if interested, I can go on, believe me ;) ). On the whole, however, out of the variety of test management tools I have used to date, I think Practitest is clean, unobtrusive and flows nicely with how I particularly like to work.

Friday, October 25, 2019

A Book Commit: MetaAutomation #30DaysOfAutomationInTesting Day Two

I am writing this series based on the requirements for the "30 Days of Automation in Testing" series as offered by the Ministry of Testing. yes, I realize I am almost 18 months late to this party but work requirements and a literal change of development environments have made this the perfect time to take on this challenge.

Let's take a look at the Day Two requirement:

Begin reading an automation related book and share something youʼve learnt by day 30.

In a sense, this is actually a re-read but a first time full apply. Several years ago, Matt Griscom approached me when I was visiting Seattle and told me about an interesting idea he had for a book. We exchanged several emails, talked about a few things here and there, I looked over some early chapters and from that (and lots of talking to other lots more qualified people than me, believe me ;) ), Matt came out with the book "MetaAutomation".

Today, that book is now in its 3rd Edition and last year Matt gave me a copy and encouraged me to give it a read. Now that we are making this product transition over to C# and .NET Core (so as to work better with other teams who are already using that stack) it seemed a very good time to take Matt up on that offer :).

Here's a bit from the Amazon description of what MetaAutomation is all about:

"MetaAutomation describes how to do quality automation to ship software faster and at higher quality, with unprecedented detail from the system under test for better communications about quality and happier teams.

This book defines the quality automation problem space to describe every automated process from driving the software product for quality measurements, to delivering that information to the people and processes of the business. The team needs this to think beyond what the QA team or role can do alone, to what it can do for the broader team. Quality automation is part of the answer to all that is broken with “test automation.”

MetaAutomation is a pattern language that describes how to implement the quality automation problem space with an emphasis on delivering higher-quality, more trustworthy software faster. Much it depends on a radical, yet inevitable change: storing and reporting all the information from driving and measuring the software product, in a structured format that is both human-readable and highly suitable to automation. This change was not possible before the technology made available by this book.

Read this book to discover how to stop pouring business value on the floor with conventional automation practices, and start shipping software faster and at higher quality, with better communication and happier teams."

I have to admit, on the surface, those seem to be bold claims and from what I read in the previous versions, there's a lot to digest here. thus, I'm actually going to be trying something out during this month. If possible, I'm going to try to approach the rest of this month's activities, where relevant, by referencing this book and trying to see if I can use the principles in my practice. Thus, in addition to doing a full form book review at the end of the month, I hope to have had a chance to actually and completely internalize what I read here. In short, Matt, I'm not just going to read this book, I'm not just going to review it, I'm going to try my best to live it. Let's see what happens if I do exactly that :).




Aedificamus: Pushing Steel and Tereré: #HealthyTech30 Day Three

Day three. Let's do this :)!!!

This is another post in my Aedificamus health and fitness series and specifically for Saron Yitbarek's #HealthyTech30 challenge. Today I will be showing what may be a blast from the past and another way to consume yerba mate. Yesterday it was the hot and traditional approach. Today will be a colder but still somewhat traditional approach :).

Exercise: Let's Push Some Steel

I'm curious if anyone out there knows what these are ;).

If you are my age or thereabouts, my guess is that you have come across these a time or two in your lives. If you are sitting there thinking "you have got to be kidding, you actually have a BULLWORKER?!" my answer is "yes, and not just one, I have two of them!"

I owned one of these when I was a teenager and I used it for conditioning when I was on my high school's track team. Years later, when I was thinking about having an easy to use at home exercise apparatus I thought, I wonder if they are still around or if I could buy a used one somewhere. Turns out they are indeed still around and thus I purchased two of them, a classic bullworker (the bigger one) as well as a smaller model called a Steel Bow.

OK, for those wondering, a Bullworker is basically a compression spring between two sleeves, with a pair of handles, one on each end and a pair of cables connected to the handles. The goal is to compress the spring either by pushing the handles or pulling on the cables. I also have a set of extension straps that allow me to do broader range movements and attach it to a door. The smaller Steel Bow model also comes apart easily and allows the user to put in one of three spring tensions (light, medium, heavy).

Today I focused on abdominals, lower back, and shoulders. Whee!!!

These devices work on the concept of isometrics, where you contract your muscles and hold them in that flexed position for an extended period, as well as a progressive resistance push. Some movements you will be able to cover a lot of range, others you will feel like you are barely moving the device at all. If you are putting pressure and able to hold it, you are getting the benefits. Point being, if you go to Goodwill you may be able to find one of these. For compact, at-home exercise equipment, this is actually pretty cool.

Food: Yerba Mate: Tereré
Thicker cut tereré leaf with a typical mate gourd
(made from calabash) and a bomba for drinking. 

Yesterday I told you all about how I've grown to love chimarrão, a variation of yerba mate that is popular in southern Brazil. It's a preparation that is, IMO, best enjoyed hot. However, I hear some of you say, what if I prefer it cold? Does it work for that too? It can be but if you like your mate cold, there's another preparation that is popular in Brazil and several other South American countries, particularly Argentina, and that's what I'll be telling you about today, tereré.

First, tereré has a different flavor profile altogether compared to chimarrão. The leaves are dried using hot air and smoke from a fire, so they have a distinctive smoky flavor to them. Additionally, the leaves are thicker, so you can prepare tereré right in a calabash gourd or you can also prepare it and drink it straight from a canning jar (granted, the second way is a bit less elegant but still works.




Traditional prep method first. This is a mate gourd and a bomba that are both well suited for drinking tereré. Preparation is simple, take about half a cup of tereré leaves and put them into the gourd. take cold water or ice water and pour it into the gourd, up to the brim. Let steep for about 10 to 15 minutes. From there, place the bomba into the gourd and drink. The bomba filters out the thicker leaves.

If you'd like to do this without having a fancy gourd and bomba, here's my other method. Again, Brazilian and South American tereré fans, my apologies if this is borderline blasphemy but hey, it works :).

1. Take a canning jar and fill it halfway with ice.
2. Put 1/2 a cup of tereré leaf into the jar.
3. Fill the jar with cold water.
4. Close the jar and shake.
5. Let sit for 10 to 15 minutes.
6. Use a tea strainer and pour off the liquid into a cup.
7. Add more cold water to the jar and keep refilling.

After multiple refills, you will reach a dilution point but it will take a while.

It is also popular to punch up tereré by adding citrus juices (lime, lemon, orange, grapefruit) or pineapple juice but it is also very nice all by itself.

How healthy these are is open to interpretation but if you are someone who actively consumes soda or other beverages throughout the day, consider switching to yerba mate and see what you think. Hot or cold, I doth fully dig it :).

Thursday, October 24, 2019

Late to the Party, Still Going to Jam: Day One, 30 Days of Automation in Testing

OK, here we go, better late than never, right :)?

Regardless, I have decided I want to tackle this one next because I am knee-deep in learning things related to code and automation and retooling with a new stack and platform.

I'm in the process of learning how to navigate my way around Windows 10 on a Lenovo Think Pad, getting my bearings with Visual Studio, and playing with some of the finer points of C# and .NET Core. In short, it's a perfect time to take this on and perhaps augment a tectonic shift in a work environment with some additional skills to help balance it all out.

This is Day One of a Thirty Days Challenge, this time focusing on Automation in Testing.

 Look up some definitions for ʻAutomationʼ, compare them against definitions for ʻTest Automationʼ.

Let's see what Lexico has to say about this (or at least what they had to say about it October 24, 2019):

automation: NOUN: mass noun

The use or introduction of automatic equipment in a manufacturing or other process or facility.

Origin: 1940s (originally US): irregular formation from automatic + -ation.


That makes sense, in the sense that that is a natural sense of the word. Something done to make something automatic. Granted, it uses the word automatic, which comes from the Greek word automatos:  acting of itself. Thus when we think of automatic, and by extension, automation, we are thinking of devices and processes acting on their own behalf. thanks, Lexico :).

So let's take a look at Test Automation. What does Lexico have to say?


No exact matches found for "Test Automation"

Interesting, and not really surprising, as it's not one word. Point is, there is no real dictionary definition for it. The closest thing we get is a Wikipedia entry (and yes, I'm a strong advocate of "caveat lector" when it comes to Wikipedia or any place else but there's nothing wrong with starting there and then extending the search. Wikipedia starts with:

"In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes." 

Where did that come from? 

Kolawa, Adam; Huizinga, Dorota (2007). Automated Defect Prevention: Best Practices in Software Management. Wiley-IEEE Computer Society Press. p. 74. ISBN 978-0-470-04212-0.

Hmmmm... not trying to be critical here but a book selling the best practices of software management gets to define what test automation means? Am I the only one who finds that somewhat amusing?

OK, to be fair, test automation is still fairly new (though it is as old as any procedural programming steps, really, so it's been with us since the 1940s at least). Ultimately it comes down to the idea that we have machines and we want to control those machines' execution and determine something about those machines' control that goes beyond them just doing that thing they do. Sorry if that's a bit breezy, I never promised you peer-reviewed scholarship ;). 

Seriously though, minus the snark, we are looking at doing a little more than repeat steps over and over. We're trying to determine a way that the steps being performed are actually being done right. thus, test automation goes a little bit beyond actual automation of steps to make a repetitive and repeatable process something a machine or pattern apparatus can do.

Aedificamus: Yoga and Chimarrão: #HealthyTech30 Day Two

I think I'm going to enjoy this feature :).

At this point, in case the hashtag picks up some new viewers, and if current viewers are wondering about the hashtag, I should probably explain for people who might be curious:

- "Aedificamus" is Latin and it (somewhat) means "building through practice". Latin aficionados will probably take exception to that but it's how I'm using it, so deal ;)!!! It is a tag and post title marker that I use here on TESTHEAD any time I talk about health or fitness.

#HealthyTech30 is an initiative that Saron Yitbarek mentioned on Twitter with the net goal that tech people like me get up and do something healthy for 15 minutes, at least, and then eat or drink something healthy and share it. If that's not screaming for blog post series, I don't know what is. Even if it isn't, I thank you all for indulging me ;).

OK, let's get to the exercise portion first.

Exercise: Fightmaster Yoga on YouTube

First off, I think anyone can benefit from doing a regular Yoga practice. Second, I particularly like Lesley Fightmaster because, though she is a good instructor, she doesn't present herself as polished or perfect. She has a laugh when her cat interrupts or people stop to watch what she is doing and that is just so human, endearing, and wonderful.

For those who want to try out her approach, she has a 30 Day Yoga for Beginners course that is terrific. I did the whole thing and yes, I would recommend. I'm currently going through her Playlist of 20 Minute workouts (because there is a lot of them and I find that length to be about perfect for me).

This is the one I did today.



Note: I have this playlist set to shuffle just to make it interesting, so I never quite know what I will end up with. You are welcome to play along with me or you can start at the beginning and work through in sequential order.

Food: Yerba Mate: Chimarrão

I hadn't consumed much Yerba Mate before my daughter moved to Brazil to serve her mission. A number of her companions are from the southern part of the country (Porto Alegre area), and they turned her on to "chimarrão" which depending on where you are is a blanket term for mate or this particular style of preparation.  She, in turn, turned me on to it and it has fast become one of my favorite beverages.

This is a unique drink with some technique and ceremony behind it. It took me a while to learn how to prepare it properly in the traditional style using a "cuia" (the gourd it is served in) and a bomba (a straw with a filter at the bottom).

Cuia and bomba. Image courtesy of Amazon.com




For those interested, I've included some pictures and an explanation of two ways to prepare chimarrão. The first is the traditional way with a cuia and bomba, the second way is with a French press.

Disclaimer: I am not pretending I know how to do this "right", just that this is the way that was communicated to me and it seems to work effectively. Also, I'm sure the French press method will be seen as borderline heresy to some. Your mileage will certainly vary :).

Traditional

For me, the traditional way is the most enjoyable because there's a set up that just looks neat and is fun to go through. First, you need a mate gourd or "cuia" that has a wide mouth and that tapers down to a more narrow neck before opening up to a larger bowl underneath. This taper is important for this preparation as I have discovered. If you have a more circular gourd, this process doesn't work quite as well.


The second item you will need is a bomba and the best way to describe this is it is a metal straw that has a strainer on the bottom to keep out larger mate leaf material. To finish the drink, you need chimarrão style mate leaf (chimarrão is a finely ground powder, I should add, though there are larger pieces of material included, at least in the kind I use).

We will also need hot water. That's it :).

First, take about half a cup of dry chimarrão and place it in the cuia. Put your hand over the end, turn the cuia upside-down and shake hard. This will compact the chimarrão mate leaf material towards the top of the cuia. This is important as the next step will help to hold it in place.

Next, take some water (about 1/3 - 1/2 a cup) and pour it into the cuia, then set it on its side, packed mate side down, so that the water will seep into the packed material. This will make a sort of "mate clay" that will hold together on the side of the cup. Take the bomba and use the wide end at the bottom to push the packed mate leaf to the side. This will make it so that the leaf stays compressed and provide a cavity so that water that you pour in later will not get "silted up" with fine ground mate leaf.

Then take hot water and pour it into the cavity. Let the cuia sit upright for a few minutes as the mate steeps. If you have done this right, you will be able to drink from the bomba and pull in the liquid mate without any of the gritty leaf material.

I love my daughter's description of the flavor of chimarrão. She says, "it's so clean and refreshing. It's like you are drinking a tree!" There is a definite leafy and earthy quality to it. Once you finish the liquid in the cup, just add more hot water. Keep refilling as often as desired. A cup's worth of packed chimarrão should last a person all day, though at some point the "pack" will start to break down and collapse back into the cuia. From experience, I will say it will take quite a while before that happens :).


French press

For those that don't want to go the traditional route (though seriously, I suggest you try it, it is so cool when you get the process down :) ), you can also use a French press the same way that it is used for preparing coffee (or similarly at least).

With a French press, the first step is to drop in about a 1/2 cup of chimarrão mate leaf. Add about a cup of hot water and stir. let sit for a couple of minutes. Add near-boiling hot water to the fill line and then give the contents anther stir. Take the top of the French press and put over the top to conserve heat. Let the mixture sit for five minutes.

Next, after letting the mate steep and settle, push the plunger on the French press and separate the leaf material from the liquid. The push-down should be fairly easy. If you feel much resistance before it reaches the bottom, wait several seconds to allow the contents to settle and push the plunger again.

If you will be pouring these into a cup to drink immediately, I suggest using a tea strainer to decant the liquid (it will stop the larger items but the really fine-grained chimarrão will pass through). From there, drink as you wish :).

I should also add that the French press method will also yield a strong enough liquid that you can keep adding hot water, stirring and repeating the pressing process all day if you wish. At some point, you will reach a maximum dilution but it will take a while, I can assure you :).

Wednesday, October 23, 2019

Aedificamus: Day One, some #HealthyTech30 Action :)

I want to say thanks to Saron Yitbarek for kicking off this idea.

She suggested on Twitter (see hashtag #HealthyTech30) that we should check-in and report on things we are doing to help us be active and eat healthy because tech workers often just sit all day.


To that end, I'm happy to kick this process off with the following:

1. My "Rebounding Workstation"

This is an interesting arrangement I came up with a few months ago so that I could have a standing workstation and add some movement to it. I have been a fan of using rebounding with various video games for years but a few months ago I decided to have a look at my work arrangement and see if I could work the benefits of rebounding. For those unfamiliar with what rebounding is, it's a miniature trampoline and the person using it can do everything from gently bounce on it to more aggressive jumping and running, as well as various balancing movements. Another nice benefit of having a rebounder is that it acts as a bit of a pump for the lymphatic system and it also helps with circulation. As a person with a serious bone-break almost a decade ago, the continued gentle back and forth really helps my lower right leg even today. Finally, unlike a treadmill desk, a rebounder is lightweight and easy to move if you need the floor space for something else (working out, yoga, stretching, strap movements, etc. More on that in future posts :).

2. My Juice Bottles

Some daily food prep. In this case, we're talking juices. I have a Breville masticating juicer and I also have a subscription to Imperfect Foods. For those not familiar, Imperfect Foods (used to be called Imperfect Produce) is a service that sells "ugly food", or in other words, food that would not be deemed "sellable" due to cosmetic issues. Perfectly good food, just not "pretty" and I'm totally OK with buying produce that may not exactly square with grocery store pretty standards. After I cut and prep everything, it really doesn't matter as long as it tastes good.

OK, so the second step to this is that I have a bunch of handled Mason jars with screw tops and straws. Each week when I get the bulk of my produce I take a bunch of it and I juice it. The resulting juice is super pulpy (which I like) and this means I am ingesting a good amount of the fiber as well. There is, of course, the leftover spent fiber that I typically freeze and use in other dishes. Waste not want not ;). I then put these juices into the jars and keep them in the fridge. I grab them for days when I'm not super motivated to cook a bunch and I just want to grab something and have it ready to go. In the mix today are:

- carrot and ginger juice (that's the orange one)

- kale, spinach, and zucchini (that's the green one)

- yerba mate made "terere" style (that the tall, dark one. I make that with a French Press, more on that another day :) ).

Tomorrow I will talk about some home exercise stuff, and maybe share some food options you can make with pomace from juicing :).


Wednesday, October 16, 2019

Two Down, Eight More to Go - Tackling More "30 Days of Testing" Challenges

As a re-enactor, a performer, and a musician, I appreciate the fact that there is a need for regular practice in any endeavor.

- While I dress up like a pirate and participate in occasional stage shows, I need to actually practice swordwork so that I can be prepared and ready, as well as SAFE, during stage performances. In short, I need to keep my body in practice with rudiments and fencing drills.

- As a musician, I can't just show up and improvise (well, I can and I have and the results have been predictably embarrassing). To be able to play at any decent proficiency and dexterity, I must practice, even if the practice I do is to play things by ear so that I can do better live improvisation. The same goes for writing songs. If I want to write better songs, I have to (gasp!) write songs in the first place. It's a little silly to think I'm going to get inspiration and write the perfect thing every time. Likewise, if I use the music theory I do know and write songs with it, I may not make something brilliant every time but my odds of writing something good go way up. Much better than if I just wait for inspiration to strike.

- When I make clothes for historical garb or cosplay, I can't just expect to come in and knock everything out the first time in perfect order. I'm just not that skilled a tailor. I can, however, make mocks and practice and try out the ideas so I can get it solid enough to make the items well.

Why should I think that as a blogger and as a tester I am just going to have intelligent things fall into my lap? The answer is "things probably won't but they definitely won't if I don't practice or prepare for them.

This brings me back to the "30 Days" Challenges. For various reasons I looked at a number of them and said "oh, that would be cool, I will check that out later" or "hmmm, not quite in my wheelhouse, I may check that out further down the road." Any guesses how many of them I've come back to? Yep, I've not come back to any of them except for the two that I chose to hit immediately. Notice that those both completed and I learned a lot from both of them. Let's have a look at a little graphic:


There are ten challenges there. Two are done, eight I've never started. Well, that's going to change. Next up is "30 Days of Automation in Testing". Why? I'm in the middle of learning how to set up C# and .NET Core for automation needs.

The problem is, we're already up to the 16th of October. Not a really convenient start time, right? Old me would say "OK, I'll start this beginning of November" and then I'd forget about doing it. I'd still feel good because I told the world I'd do it. I mean, who is going to check up on me, right? Well, that's a lame attitude and the answer is I'M GOING TO CHECK UP ON ME!!! 

By the way, expect me to talk about "Writing While ADHD" but I'm not going to promise a timeline for it just yet ;).

So what's my plan for the "30 Days of Automation in Testing"? Simple, I'm starting it today. Seems two posts a day should be enough to get me back on track and cover 30 days (that may be aggressive and ambitious but hey, fools rush in where consultants fear to tread ;) ).

Am I Really So Ordinary? - a #PNSQC2019 Blog Retro

This has been an amazing few days. I've received a presentation award from my peers here. You all voted with your evaluations and you felt my score warranted the second most highly rated presentation of the conference. WOW!

I'm humbled by this but I'm also a little embarrassed. Why would I be embarrassed? Because for the past five months, my blog has been quiet. Why is that? Because I've felt that I don't have anything important left to say. TESTHEAD has been on the air for almost ten years. There are over 1200 blog posts I have written. What more could I possibly say without repeating myself? What can I possibly add that would be even remotely interesting?


I don't know if anyone else has these thoughts from time to time... or often... or every single day... but yeah, I do. I had a great conversation last night with a fellow presenter (they may or may not be cool with me sharing this so I'm cloaking in a little anonymity... but I'm pretty sure anyone who knows us can guess ;) ). As I was talking about how I struggled to come up with an idea this year and that I wondered if my experience would even be all that interesting, we recapped a few things and thoughts:

- experiences are all we can really share and there what people actually relate to. Me setting myself on high and offering pronouncements is boring. Me telling how I got completely lost or frustrated with a situation and what I learned from it is much more valuable.

- I joked that so much of my talk was "blinding flashes of the obvious" and the response back was "was it really? If it was so obvious, why was it a revelation when you addressed it?" Point being, what may seem patently obvious in hindsight may be hidden or not understood by everyone else. In short, if you are confused, it's a good bet a lot of other people are, too.

- it takes a lot to get people to get up on a stage or in front of a group of people to be willing to speak. What we may see as banal and every day is a major step out of the comfort zone for 95% of people. The act of presenting is courageous in and of itself, much less someone willing to do it again and again, year after year.

- what's more, think about what people do to agree to come to a conference in general. They give up their time, their families, their work commitments, their home commitments, many of them pack themselves into a plane for several hours and are not at all thrilled about the experience, yet they go because they want to hear what might give them an edge, a new idea, a new angle to help them do better work every day. They want to hear what you have to say, and really, the only worthwhile thing that you can share is your own experiences.

I should also mention that the thoughts for my talk didn't come together fully formed. The paper I submitted went through three revisions and extensive feedback from two other individuals that helped me take ideas that were half baked and get them to make more sense, as well as to be able to step back and help me emphasize the areas that needed to be and push back or disregard areas that didn't add as much as the parts they suggested I emphasize. For those who voted for my presentation, I must be absolutely clear that "I HAD HELP!"

Others have asked if I will be back next year and what I might talk about? The answers are "likely, yes!" and "I really do not know at this stage" but I have a few ideas. One thing I want to do is go back and review the other "30 Days" challenges that the Ministry of Testing has put together. I have several areas in my own work environment that is requiring me to step out of my comfort zone (have I mentioned I'm trading in my MacBookPro for a Windows 10 machine? Have I mentioned that I'm looking into what it takes to program in C# and run on .NET Core? Yeah, those are new realities for me. If you asked me last week, I might have said "yeah, no one is really interested in that." Today? I have a totally different opinion on that front. I'm still learning things and there's a lot to learn so it only seems reasonable I keep learning in public the way I've said I would :).

PS, I've been on a voyage of musical discovery with my younger daughter recently and part of that has been to introduce singer/songwriter Paula Cole to her. Today's title borrows from her first single "I Am So Ordinary" so credit where credit is due ;).


A Show of Hands - a #PNSQC2019 Live Blog

Today is the workshop day at PNSQC. I'm the moderator for Melissa Tondi's workshop on "Efficient Testing". As the workshops are an add on and paid for by the attendees, out of respect for that I do not live blog workshop goings-on; If you want to take part, come out and sign up to be a part of it ;).

Instead, I' like to talk a little bit about what I think really makes PNSQC unique, and that is its emphasis on working with volunteers.


  • If you submitted a paper and you received feedback, that person providing feedback is a volunteer.
  • If you interact with the web site, those updates are done by volunteers.
  • The registration, room monitoring, moderating of tracks, etc. are done by volunteers.

In short, this conference has so many opportunities to volunteer and participate. Many of the opportunities available will get a person a free ticket to the conference. Volunteering for workshops also gives a person the opportunity to participate in that workshop for free. While there is no guarantee that a person will be able to moderate or facilitate the specific workshop they want to participate in, odds are still pretty good that if you show interest early, you can moderate your first choice.

The bottom line here is that the conference is an excellent one, IMO, and the volunteers go a long way in helping foster that experience. 



Tuesday, October 15, 2019

Testing AI and Bias - a #PNSQC2019 Live Blog

Wow, have we really gotten to this point already? We're down to the last formal talk, the last Keynote. These conferences seem to go faster and faster every time I come out to play.

I've had the chance to hear Jason Arbon talk a number of times at a variety of conferences over the past several years and Jason likes to tackle a variety of topics with ML and AI. Thus, I'm definitely interested to see where Jason was going to go with the topic of AI and Bias. This is a wild new area of testing and as many of you all know I am fond of Carina Zona's talk "Consequences of an Insightful Algorithm".

OK, we understand bias is there but what can we actually do about it? Well, here's the spoiler. You can't remove bias from AI. That is by design. AI takes training data and based on the training system it learns. Literally at the start of the process bias enters in. The point is not to eliminate bias, it is to make sure that undesirable bias is not present or is minimized.

Think of a search engine. How can a machine look at a number of source articles and then, based on a query, decide what might be the most important information? We start with an initialized system. Think of it as a "fresh brain" and not in the zombie sense ;). From there, we then go to a training system, which is information already graded and scored by a human or group of humans, so that the system can train on that data and those values. Can you guess where the bias has crept in? Yep, it's there with the training set. If the system guesses wrong, it gets negative reinforcement. If it gets it right, it gets positive reinforcement (from a machine's sense of reward, I guess).

There are other factors at play such as commerce (ads will get preference or at least money will). Crawlers start on "popular" sites and then look at less linked sites. It will also be biased, by language, education and very likely dominant gender and race. There is also the fact that Microsoft Bing is set by default for Windows. For those people who don't know how to change their search engine, you end up with a "lowest-common-denominator" of users that match up with some weird demographics. According to Jason, Microsoft Bing's core user group is single women in the midwest (he said it, not me :p ). Indicative of that is that much of Microsoft Bing's largest search volume comes from that demographic. What might that indicate what is "feeding the neural network"?

There is also Temporal Bias or "Drift". Over time, the searches can be effective by many things such as weather, politics, astrology, etc. Sample Size can also affect the way that data is represented. One way to check this is to keep changing the sample size until the scores stop changing. Not a guarantee but at least it gives a better feeling that more people are being represented.

There is also a bias in the literal training data. In most cases, the people who provide seach data training sets are for people paid $20/hr or less. In short, the people feeding are neural networks are not those of us who are designing them. We can debate if that is a good or bad thing or if software engineers would be any better.

There's even a Cleaning Bias. Bing cleaned out mis-spellings, random numbers and letters, etc.and the irony was that Google didn't do that and thus Google can even help people find ht they are looking for even if they misspell it.

What happens when there is no "right" answer? Right answer meaning there is one answer for a particular word and there are multiple possible different options for the same word?

As Jason has said a few times, "are you all scared yet?!" Truthfully, I'm not scared but I am a lot more cynical. Frankly, I don't consider that a bad outcome. I think it helps to have a healthy skepticism of these types of systems. We as testers should remain relevant a bit longer if we continue to be ;).