Exploratory Testing Driven by Mind Maps with @claubs_uy (#PNSQC2021 Live Blog)

 

When it is applied correctly, in a targeted and purposeful manner, Exploratory Testing can be an effective and useful tool to any tester for learning, discovering new avenues of using the product (both intended and unintended), and developing additional questions for further investigation (in short, Exploratory Testing is the application of good science, IMHO).

Mind maps are a neat way to get ideas down on paper (and to be clear, paper or a whiteboard is much more my preferred method of creating them). I've worked with a few different versions on a computer but I will confess I have often struggled with using them. that's mostly a personal problem, as I know many people who are well versed and like using the software programs for mind maps. 

So what do these two things have in common? I have my own thoughts on that, of course, as I think that mind maps can be a great way of organizing thoughts to create initial charters and track what I am doing as I am doing it... but what does Claudia Badell have to say about it? that's what I'm here to find out :).

In my world view, Exploratory Testing is a way to focus on learning about the system and its dependencies, ways to interact with the system, and in the process, feel out the limitations the systems may present to us and how we might be able to exploit those limitations to find potential vulnerabilities. If we are doing this by ourselves, or we are the lone tester on a team, then our exploration and a report of what we find might be sufficient. How about if we need to work with a broader team and testing is a common activity amongst the entire team? This is where lessons learned brain dumps or report is not necessarily sufficient and having an additional "information radiation" method would be warranted and desired. In this case, Mind maps and specifically online mind maps would be a good tool to share with others. Okay, deep breath, guess my paper strategy isn't going to work here, but that's fine :).

Claudia recommends building a mind map for each feature or each story, and have them contain test conditions, ideas, and variables to cover for the specific feature or story.

As we see each of these nodes created, each of those nodes could effectively be an Exploratory Testing session. For that matter, we could also create sessions for each leaf coming off of a node, such as we see below:

Now something I think could be cool (and it may be available, I just haven't used it) would be a way to attach an exported text file or image file for each leaf node. this way we could gather our notes from Exploratory test sessions and incorporate them within the mind map if desired (at least I could see that as a benefit, others mileage might vary :) ).

Claudia suggests the idea of a meta-language so that we have a clear understanding of how the ideas are represented. Consider this sort of like a Gherkin for Mind Maps but not necessarily that strict. The metal language doesn't just mean words, it could mean colors, symbols, images, and branching. The point, as I'm hearing it, is not so much what the meta-language is but that the org as a whole understands it so that everyone who would use it can use it.

So overall the goal of the mindmaps is to be a reference for points to start and avenues to consider for Exploratory Testing sessions. Each session will be unique but we need not reinvent the wheel every single time (as appealing as that sounds at times ;) ). Instead, the Mind Map allows us to focus on developing hypotheses, and our exploratory sessions are our experiments. the data we gather can then be included with or imported back into the mind map and now we have a collection of values we can use again later. Thus the mind map isn't just a once-and-done item. Instead, it is continually updated and reused. Cool, this is already getting me a little more excited about moving away from my preferred paper method.

One of the lessons Claudia shared is that having the ability to reference and notate a shared mind map and keep exploratory notes is that the team as a whole can improve and update the testing ideas and testing strategy (see my comments yesterday on the value of a less formal test strategy; this seems like it might fit that bill beautifully). Also, I like the idea of iteratively bumping up and adding/pruning information so that we are dealing with what is pertinent and important for each feature rather than just having an ever-growing list of busywork to deal with. 

I have to admit, I entered curious and I'm leaving with a concrete plan as to how I can implement this with no permissions needed. I love when I get those, so thank you, Claudia :).

Concentration on Information Radiation

Ever have an experience at a conference that just sticks with you, that one thing that seems like such a little thing and then, as you consider it more and more, you just slap your forehead and say to yourself "for cryin' out loud, why didn't I think of that?!"

On Monday while I was at CAST, I was the room helper for Dhanasekar Subramaniam's tutorial about "Using Mind Maps for Mobile Testing". Much of the session was around heuristics for mobile testing and the ability to capture heuristics elegantly inside of mind maps. as part of the process, we spent a bit of time creating mind maps in XMind to use later with our chartered test sessions. I've done this before. I've even created a full mind map of the entire James Bach Heuristic Test Strategy Model (yes, one mind map and yes, when fully expended it is massive. Probably too massive). As we were working to create nodes and sub-nodes, Sekar pointed out that there were many labels that could be applied to the nodes, and that the labels were additive. In other words, each node could have several labels applied to them. 

As I was looking at this, and seeing labels such as pie chart fill, green check boxes, people silhouettes of several different colors, red x's, green yellow and red explanation points, and many others, I started thinking about how, by color and proximity, we could gauge how much coverage we have given a particular mindmap (or in this case, how completely we have applied a heuristic to testing) and what the results were for that. Instead of stopping to write down lots of notes, each node we were testing would get a label placed, and each label would have a semantic meaning. Green check box meant good, red X would mean failed or something wrong, a quarter pie chart would mean a quarter done, a yellow square would mean something that was a warning, but maybe not an error. Different color people icons would mean the person who performed that set of steps, and so on.

As I was looking at this, I joked with Sekar that we could tell the entire testing story for a feature or the way we applied a heuristic to a story in one place with one relatively small map. We both chuckled at that, and went on to do other things.

The more I thought about this, though, the more I liked the idea. In a previous company, we set up a machine and had a couple of flat screen monitors attached. These flat screens were placed in the main area and left on, cycling the images that were shown, only in this case, they were graphs and pages of results that were relevant to us. In short, they were acting as information radiators for our team. At a glance, we could know if the build had failed, if deployment was successful or not, and where the issue would be if there was one. We could also use this technique for information radiation. Imagine a charter or set of charters. Each one had their own mind map. Each mind map could be cycled through presentation on the monitor(s). the benefit would be that, at a glance, the team would know how testing was going fort that area, and we could update it all very quickly. I kept experimenting with it, and the more I did, the more I became convinced this just might work.

To that end, I am holding a Weekend Testing session this coming Saturday, August 8, 2015 at 10:00 a.m. PDT. We will look at mind mapping in general and XMind in particular, and we will develop a small heuristic for a feature (within XMind itself) to test and to update. I really like this idea, but I want to see if it can be tinkered with, and if it might be a useful approach to others.

If you think this might be a fun way to spend a couple of hours, come join us on Skype. Contact "weekendtestersamericas" and add us as a contact. On Saturday, get on Skype about 20 minutes before the session and say you want to be added to the session. Prerequisite, if you want to follow along and actually do the exercise, would be to download the XMind app for your platform.

 Again, I apologize for the short notice, but I hope to see you Saturday.

It's the Map of the Game - Live from #CAST2015

One of the interesting approaches we experimented with was a game called "The Room" which has a slightly Silent Hill puzzle vibe to it, sans the homicidal faceless nurses, of course ;). We spent the better part of a half hour exploring the game and learning about the details we can play with in its tutorial setting.

What was the point to this? Well, other than getting us all to play a game for thirty minutes, we had a chance to consider how we interact with mobile devices and the methods of interaction. I remember when the iPhone was released in 2007, and while I didn't jump on and buy one. I was familiar with them from testing, and it was a definitely different way to interact with a device. Think of all the ways that you interact with an iPhone or an Android device (and sure, A Windows Phone, too, though I've never actually used one). The touch and multi-gesture format, the loading of apps, and the mobile first model is making some interesting changes to how we consume information and the way that sires work. Sekar made the point that a number of businesses in India are doing everything through mobile... they are literally shutting down and turning off their traditional desktop web sites!

Interacting with mobile devices brings some fundamental challenges that working on a desktop or laptop device do not. Beyond touch alone, there are issues of the screen space and how characters are displayed, as well as the input options and how the user interacts with the product. Unlike a traditional web site, where data entry at a QWERTY keyboard is required, mobile apps are meant to be optimized for minimal typing and movements. Many updates and data items are created by using the accelerometer. or via geolocation, or through other methods. In other words, orientation and motion affect how an app works (in some games, orientation and motion are specific actions required for the games to be played).

One thing I have learned from testing is that the battery state has a lot to do with the way that a system responds. Systems work better with phones that have a greater charge as compared to those that have a lower charge. Screen orientation can greatly affect what the app displays (horizontal vs. vertical). What do we see when we switch from horizontal to vertical? Data plans can vary wildly, and the amount of data we can use also varies. To allow us to leverage WiFi hotspots, the phones are able to jump onto known networks whenever we are in the vicinity, provided we have met the requirements for authentication. How does our app respond when it jumps from cellular data network to WiFi and back?

This is the point where mind maps and mobile testing can come together. Mind Maps are actually accessible and modifiable via mobile devices, more so that large text documents with a lot of text are. By using a mobile device while we test it, we can capture interesting observations and update a mind map via a mobile device more quickly than locating a desktop computer or laptop and making the updates.

Sekar provided us with a detailed mind map that shows some of these considerations, so we're going to go back and reconsider some of the tests we performed on Hop! with this in mind, and utilize the map to help guide our efforts... indeed, we are much more focused and targeted with our testing and exploration this time around (I guess it really does help navigation when you have a good quality map :) ).

A CAST of 1,000 - Live from #CAST2015

I'm smiling a little bit today as I have realized that this is a special post for me. This particular post is number 1,000 on Testhead. When I started this blog, I don't think I ever imagined I would get to 1,000 posts, but I am still here, and you are still reading, and both of those realities make me very happy. Forgive the title, but it's just a quick celebratory dance, and on with the show.

We've been focusing on mind maps during the morning in Dhanesekar Subramanian's Tutorial on "Mobile App Coverage Using Mind Maps". The first part of the morning was spent discussing how to create mind maps and the properties of what makes a mind map effective. During this part of the morning, we are looking at some ways that a mind map can be use to develop a testing strategy and actually tracking coverage and progress. In addition to creating topics and sub topics, we can create markers that have semantic meaning, and we can also create nodes that have markers that show pass or fail, or can show level of completion:

A slightly silly example, but this shows how you can layer images on a node to convey information.

This is an interesting avenue for communicating exploratory test ideas. as I was playing with the admittedly simplistic music mind map, I realized that this could be a way to communicate a research coverage for a topic, or to be used for making quick notes for application test coverage. Instead of having to read through a bunch of minute details, especially if the testing is mostly clean with little in the way of comments. If we have sub-sections that can be marked as underway, completed, or those with issues, then I can drill down to see the issue and defocus on the rest of the stuff that is clean.

As an exercise, we are asked to look at a Mobile app from AirFrance called "Hop!" One of the areas that we initially started looking was to take the app and split it into the major functions on the screen (Book a Flight, Check an Itinerary, etc.) As we were looking at the workflows available to us, the natural reaction was to go through and break down the steps of the workflow into their own nodes, and give each option a node of its own. What we realized by doing this was that we made a lot of branches and separate paths for individual tests... our mind map quickly became cluttered with step details, and we had only looked at one button. If we were to do this for every option, we would have a very detailed and very messy mind map to try to look at. Instead, we considered looking at each workflow and display each workflow as a single node, at least to start with. The benefit to this is that this allows us to give the app a broad view first, and to create the important nodes that are the essential interactions. There will certainly be branches we can drop into, but rather than chasing down minutiae of steps, we can word the nodes so that they are unique workflows without the need to atomic specificity.

One of the things I am realizing with mind maps is that there are ways that popping a node can change the purpose of the map. By adding a "heuristics" node, we can turn a mind map that is a static model of an application into a test strategy, and do so with very little work. Even with these discoveries and neat uses, the biggest challenges with mind maps, unless they are very basis, is the fact that communicating the meaning of the map will take some communication. I've had mind maps sent to me as notes from a talk or a presentation that were very clear and understandable, and I've received mind maps that were undecipherable, at least to me they were. However, when I started talking to the person who made the map, and got to understand their individual context or the way that they were communicating the information, then the system became clear, and I understood the reasoning behind what they were recording.

It's time to take a break for lunch, so I'll chat with you all again in an hour or so. I hope you are finding this as fun as I am :).

CASTing my Line into Mind Maps - Live from #CAST2015

Today the rubber meets the road. We are going live today with CAST 2015, breakfast has happened, the setup for all of the bags and badges has happened, the tables are manned, the food is being served, and the prep work for the webCAST is happening now (which I do not have any real involvement with other than to be grateful that it is happening and that Ben, Dee and Paul are making it happen).

Monday at CAST is tutorial day, and since the participants pay extra for the content, I have traditionally not done a play by play of these sessions, but I will talk a bit about why I chose to be in the one I am in and what my role is in being here. I'll also talk a bit of my own observations around the topic without repeating Sekar's presentation. Also, to be kind to my readers, I will split these posts up into different posts, since many have said it's hard to follow a big long post throughout the day.

First off, each of us who are on the board and attending a tutorial are doing so as the room helper or facilitator for the tutorial. One of the things I asked all of us on the board who wanted to participate to do was to work the back channels for the event. Since there was a limited number of seats for each tutorial, we wanted to make sure that the participants in the conference go the first chance to be there. We also made sure that each of us picked a different tutorial to be part of so that we would be able to evaluate the individual sessions and be able to report back from each of them as to their effectiveness, and things we could learn to help the next year's organizers with choosing and developing solid sessions.

There are four sessions being offered this year for tutorials:

Christin Wiedemann is leading a tutorial called "Follow your Nose Testing - Questioning Rules and Overturning Convention

Dhanasekar Subramanian is leading a tutorial called "Mobile App Coverage Using Mind Maps"

Robert Sabouring is leading a tutorial called "Testing Fundamentals for Experienced Testers"

Fiona Charles is leading a tutorial called "Speaking Truth to Power: Delivering Difficult Messages"

Since we all opted to spread ourselves around the tutorial choices to be the room helpers and facilitators, I chose to work with Sekar and be part of the "Mobile App Coverage Using Mind Maps" tutorial. One of the reasons I chose to focus on this tutorial was that I have seen a variety of mindmaps used by people over the years, and I tend to focus on a fairly simplistic use of them. For me, I tend to use a core concept, and branch a few ideas off the core, and then break them down to a few words in the branches. If I need more depth, rather than make complex mind maps, I will usually just create a new mind map with another concept. the idea of having multiple branches on the same map just feels messy to me, but at the same time, having to jump to multiple maps is also messy.

Sekar's tutorial is covering two concepts at the same time. The first is giving participants a chance to work with mind maps who may not have done so in the past. The second part is testing with mobile apps and categorizing the details of the app. the benefit to using Mind Maps in the process of testing is less the rigid use of the tool and more of idea that each core concept can have several points where we can branch off.

One of the fun things we do in these tutorials is get everyone on the same page and playing with the discussions. In this tutorial we are encouraged to break into groups and discuss a variety of topics. For mine I chose music, and what I find interesting with mind maps is not so much what we add to the mind map, but why we add them. My map for music was broken into Instruments, Songs, Genres and Modes. Why did I choose those? Possibly because I am a musician, and those are the things I think about. My guess is that a casual listener might not even know that modes exist, so they wouldn't include it as part of their breakdown.

It's time for our morning break, so I'm going to call and end to this post. I'll be back with another one in a bit. Thanks for joining me today :).

Create a MindMap: 99 Ways Workshop #18

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.

My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions.

Suggestion #18: Create a MindMap - Rosie Sherry

Here's a specific skill that can be worked on and achieved through a number of different means. A Mind Map (or mind mapping) is a physical way to capture a mental model and help organize thoughts and ideas.

Image courtesy the Wikipedia article on Mind Mapping

The idea behind a mind map is that you start from a central word or concept, and you branch out into words or concepts that associate with that initial element. Each of these nodes can, additionally, have sub-nodes that you can associate additional ideas and words with (and so on, and so on and so on).

Mind maps can be done on paper, on a white board, or with any drawing application on a computer or a mobile device. There are also dedicated mind mapping applications available. A benefit of these dedicated tools is that the user can expand or collapse branches and sub branches,focusing on the areas that are most important at the given moment. One example of a free tool that is available for mind mapping (and the one that I use) is Xmind. I'm referencing it for convenience sake. There's plenty of others out ther. Do a search for "Mind Mapping tools" and you will see what I mean.

Workshop #18: Practice using Mind Maps when approaching concepts or challenges

To get in the habit of using mind maps, the best way to do it is to start small. Think of something you would like to organize information around.

Start with a central idea or topic.

Example: create a central node around the term HICCUPPS

Then consider the various areas that you can branch off that idea. 

History

Image

Claims

Comparable products

User expectations

Product

Purpose

Statutes

From there, look at each node/branch. 

Could you add details for each of the terms?

What sub-branches could you create based on those terms?

Do some of the sub-branches relate to others? If so, make a representation that shows that they can be inter-related (usually by drawing a dotted line from one to another).

By the time you finish, you will have a relatively compact model with some key ideas in a small space. Yes, I'm deliberately not drawing an example map here, because I want each of you reading this to try it out for yourself, and try out ways that work best for you.

One benefit of using a software tool that is designed for making mind maps is  that you can make truly large maps that, can capture a lot of information in a single place. Having the ability to collapse or expand nodes makes even large concepts manageable. As an example of a "Big Map", I took James Bach's Heuristic Testing Strategy Model and made it into a single mind map, so that I could use it with a variety of testing situations. For those interested in playing around with it, it's at http://www.xmind.net/m/WhWe/

Bottom Line:

Mind maps can be effective ways of organizing thoughts and making connections between ideas and putting ideas down quickly. It allows for a way to communicate a lot of information without taking a lot of space or words to do it. They can be as simple or as complex as you want to make them. The key is to get in the habit of working with them and using them to capture and convey your ideas.