Three blocks down one to go :).
This has been a productive and fun day, and I want to say thanks to Dhanasekar Subramanian for putting together an entertaining and informative session.
As we left the third block to fill up on Diet Mountain Dew and cookies (well, that's what I did, I really can't speak for the rest of the participants) we were looking at utilizing a mind map to sketch out the application and look at testing artifacts that we find. That's pretty cool in and of itself, but what about the next project? What could we do to consider and focus on a totally different app?
Truth is, we don't want to re-invent the wheel, but there are a number of key areas that we can ask "what if?" questions about. Instead of making a list of specific questions to make lots of specific mind maps, it can be helpful to have some common "rules of thumb" to draw upon. If you are reading that and want to yell "your honor, Testhead is leading the witness", well, yes, I am. For a lot of you, this is going to seem like a blinding flash of the obvious, but for those who are not familiar with the term, this is where heuristics come into play. Heuristics are wonderfully suited for mindmaps. Sekar in fact has written about, and uses in his tutorial, a good heuristic for testing mobile app coverage.
LONG FUN CUP
Below are the quick and dirty descriptions that Sekar uses to describe these terms. The "sins" are straight from his blog, and they get the point across, methinks ;):
Location: It’s a sin to test mobile app sitting at your desk, get out!
Orientation: It’s a sin to test mobile app sitting at your desk, lie in the couch.
Network: It’s a sin to test mobile app sitting at your desk, switch networks.
Gestures: In the mobile world, app responds to gestures, not clicks.
Function: Does the application fulfills core requirements?
User scenarios: How easy or how hard is it to complete a task using the app?
Notifications: How does the app let us know something needs our attention?
Communication: How does the app behave after interruptions by an incoming call or an SMS?
Updates: How does your device handle updating apps? What happens when we do?
Platform: Why does Apple and/or Android do certain things in a certain way?
What I like about taking a heuristic and turning it into a mind maps is the fact that you can communicate a particular testing strategy up front and very quickly. LONG FUN CUP contains a lot of potential testing horsepower if it is thoughtfully applied. What can also add to the ability to quickly communicate information is using the labels, tags and other icons to help communicate information quickly. In this case a display of a mind map with the icons for each area can be a quick information radiator. the areas without icons can be seen as areas that still need to be addressed. areas with progress icons can show how much is done. Green check boxes can show that areas pass or are at least not seen to be having issues at this time. red X marks or exclamation alerts can point to potential problems, and text boxes can be filled in with more details or pointers to other documents that provide greater depth. What's more, with the right tools, doing these updates could be done on the mobile devices themselves, making for a nice virtuous cycle.
Monday, August 3, 2015
It's the Map of the Game - Live from #CAST2015
One of the interesting approaches we experimented with was a game called "The Room" which has a slightly Silent Hill puzzle vibe to it, sans the homicidal faceless nurses, of course ;). We spent the better part of a half hour exploring the game and learning about the details we can play with in its tutorial setting.
What was the point to this? Well, other than getting us all to play a game for thirty minutes, we had a chance to consider how we interact with mobile devices and the methods of interaction. I remember when the iPhone was released in 2007, and while I didn't jump on and buy one. I was familiar with them from testing, and it was a definitely different way to interact with a device. Think of all the ways that you interact with an iPhone or an Android device (and sure, A Windows Phone, too, though I've never actually used one). The touch and multi-gesture format, the loading of apps, and the mobile first model is making some interesting changes to how we consume information and the way that sires work. Sekar made the point that a number of businesses in India are doing everything through mobile... they are literally shutting down and turning off their traditional desktop web sites!
Interacting with mobile devices brings some fundamental challenges that working on a desktop or laptop device do not. Beyond touch alone, there are issues of the screen space and how characters are displayed, as well as the input options and how the user interacts with the product. Unlike a traditional web site, where data entry at a QWERTY keyboard is required, mobile apps are meant to be optimized for minimal typing and movements. Many updates and data items are created by using the accelerometer. or via geolocation, or through other methods. In other words, orientation and motion affect how an app works (in some games, orientation and motion are specific actions required for the games to be played).
One thing I have learned from testing is that the battery state has a lot to do with the way that a system responds. Systems work better with phones that have a greater charge as compared to those that have a lower charge. Screen orientation can greatly affect what the app displays (horizontal vs. vertical). What do we see when we switch from horizontal to vertical? Data plans can vary wildly, and the amount of data we can use also varies. To allow us to leverage WiFi hotspots, the phones are able to jump onto known networks whenever we are in the vicinity, provided we have met the requirements for authentication. How does our app respond when it jumps from cellular data network to WiFi and back?
This is the point where mind maps and mobile testing can come together. Mind Maps are actually accessible and modifiable via mobile devices, more so that large text documents with a lot of text are. By using a mobile device while we test it, we can capture interesting observations and update a mind map via a mobile device more quickly than locating a desktop computer or laptop and making the updates.
Sekar provided us with a detailed mind map that shows some of these considerations, so we're going to go back and reconsider some of the tests we performed on Hop! with this in mind, and utilize the map to help guide our efforts... indeed, we are much more focused and targeted with our testing and exploration this time around (I guess it really does help navigation when you have a good quality map :) ).
What was the point to this? Well, other than getting us all to play a game for thirty minutes, we had a chance to consider how we interact with mobile devices and the methods of interaction. I remember when the iPhone was released in 2007, and while I didn't jump on and buy one. I was familiar with them from testing, and it was a definitely different way to interact with a device. Think of all the ways that you interact with an iPhone or an Android device (and sure, A Windows Phone, too, though I've never actually used one). The touch and multi-gesture format, the loading of apps, and the mobile first model is making some interesting changes to how we consume information and the way that sires work. Sekar made the point that a number of businesses in India are doing everything through mobile... they are literally shutting down and turning off their traditional desktop web sites!
Interacting with mobile devices brings some fundamental challenges that working on a desktop or laptop device do not. Beyond touch alone, there are issues of the screen space and how characters are displayed, as well as the input options and how the user interacts with the product. Unlike a traditional web site, where data entry at a QWERTY keyboard is required, mobile apps are meant to be optimized for minimal typing and movements. Many updates and data items are created by using the accelerometer. or via geolocation, or through other methods. In other words, orientation and motion affect how an app works (in some games, orientation and motion are specific actions required for the games to be played).
One thing I have learned from testing is that the battery state has a lot to do with the way that a system responds. Systems work better with phones that have a greater charge as compared to those that have a lower charge. Screen orientation can greatly affect what the app displays (horizontal vs. vertical). What do we see when we switch from horizontal to vertical? Data plans can vary wildly, and the amount of data we can use also varies. To allow us to leverage WiFi hotspots, the phones are able to jump onto known networks whenever we are in the vicinity, provided we have met the requirements for authentication. How does our app respond when it jumps from cellular data network to WiFi and back?
This is the point where mind maps and mobile testing can come together. Mind Maps are actually accessible and modifiable via mobile devices, more so that large text documents with a lot of text are. By using a mobile device while we test it, we can capture interesting observations and update a mind map via a mobile device more quickly than locating a desktop computer or laptop and making the updates.
Sekar provided us with a detailed mind map that shows some of these considerations, so we're going to go back and reconsider some of the tests we performed on Hop! with this in mind, and utilize the map to help guide our efforts... indeed, we are much more focused and targeted with our testing and exploration this time around (I guess it really does help navigation when you have a good quality map :) ).
A CAST of 1,000 - Live from #CAST2015
I'm smiling a little bit today as I have realized that this is a special post for me. This particular post is number 1,000 on Testhead. When I started this blog, I don't think I ever imagined I would get to 1,000 posts, but I am still here, and you are still reading, and both of those realities make me very happy. Forgive the title, but it's just a quick celebratory dance, and on with the show.
We've been focusing on mind maps during the morning in Dhanesekar Subramanian's Tutorial on "Mobile App Coverage Using Mind Maps". The first part of the morning was spent discussing how to create mind maps and the properties of what makes a mind map effective. During this part of the morning, we are looking at some ways that a mind map can be use to develop a testing strategy and actually tracking coverage and progress. In addition to creating topics and sub topics, we can create markers that have semantic meaning, and we can also create nodes that have markers that show pass or fail, or can show level of completion:
This is an interesting avenue for communicating exploratory test ideas. as I was playing with the admittedly simplistic music mind map, I realized that this could be a way to communicate a research coverage for a topic, or to be used for making quick notes for application test coverage. Instead of having to read through a bunch of minute details, especially if the testing is mostly clean with little in the way of comments. If we have sub-sections that can be marked as underway, completed, or those with issues, then I can drill down to see the issue and defocus on the rest of the stuff that is clean.
As an exercise, we are asked to look at a Mobile app from AirFrance called "Hop!" One of the areas that we initially started looking was to take the app and split it into the major functions on the screen (Book a Flight, Check an Itinerary, etc.) As we were looking at the workflows available to us, the natural reaction was to go through and break down the steps of the workflow into their own nodes, and give each option a node of its own. What we realized by doing this was that we made a lot of branches and separate paths for individual tests... our mind map quickly became cluttered with step details, and we had only looked at one button. If we were to do this for every option, we would have a very detailed and very messy mind map to try to look at. Instead, we considered looking at each workflow and display each workflow as a single node, at least to start with. The benefit to this is that this allows us to give the app a broad view first, and to create the important nodes that are the essential interactions. There will certainly be branches we can drop into, but rather than chasing down minutiae of steps, we can word the nodes so that they are unique workflows without the need to atomic specificity.
One of the things I am realizing with mind maps is that there are ways that popping a node can change the purpose of the map. By adding a "heuristics" node, we can turn a mind map that is a static model of an application into a test strategy, and do so with very little work. Even with these discoveries and neat uses, the biggest challenges with mind maps, unless they are very basis, is the fact that communicating the meaning of the map will take some communication. I've had mind maps sent to me as notes from a talk or a presentation that were very clear and understandable, and I've received mind maps that were undecipherable, at least to me they were. However, when I started talking to the person who made the map, and got to understand their individual context or the way that they were communicating the information, then the system became clear, and I understood the reasoning behind what they were recording.
It's time to take a break for lunch, so I'll chat with you all again in an hour or so. I hope you are finding this as fun as I am :).
We've been focusing on mind maps during the morning in Dhanesekar Subramanian's Tutorial on "Mobile App Coverage Using Mind Maps". The first part of the morning was spent discussing how to create mind maps and the properties of what makes a mind map effective. During this part of the morning, we are looking at some ways that a mind map can be use to develop a testing strategy and actually tracking coverage and progress. In addition to creating topics and sub topics, we can create markers that have semantic meaning, and we can also create nodes that have markers that show pass or fail, or can show level of completion:
 |
| A slightly silly example, but this shows how you can layer images on a node to convey information. |
As an exercise, we are asked to look at a Mobile app from AirFrance called "Hop!" One of the areas that we initially started looking was to take the app and split it into the major functions on the screen (Book a Flight, Check an Itinerary, etc.) As we were looking at the workflows available to us, the natural reaction was to go through and break down the steps of the workflow into their own nodes, and give each option a node of its own. What we realized by doing this was that we made a lot of branches and separate paths for individual tests... our mind map quickly became cluttered with step details, and we had only looked at one button. If we were to do this for every option, we would have a very detailed and very messy mind map to try to look at. Instead, we considered looking at each workflow and display each workflow as a single node, at least to start with. The benefit to this is that this allows us to give the app a broad view first, and to create the important nodes that are the essential interactions. There will certainly be branches we can drop into, but rather than chasing down minutiae of steps, we can word the nodes so that they are unique workflows without the need to atomic specificity.
One of the things I am realizing with mind maps is that there are ways that popping a node can change the purpose of the map. By adding a "heuristics" node, we can turn a mind map that is a static model of an application into a test strategy, and do so with very little work. Even with these discoveries and neat uses, the biggest challenges with mind maps, unless they are very basis, is the fact that communicating the meaning of the map will take some communication. I've had mind maps sent to me as notes from a talk or a presentation that were very clear and understandable, and I've received mind maps that were undecipherable, at least to me they were. However, when I started talking to the person who made the map, and got to understand their individual context or the way that they were communicating the information, then the system became clear, and I understood the reasoning behind what they were recording.
It's time to take a break for lunch, so I'll chat with you all again in an hour or so. I hope you are finding this as fun as I am :).
CASTing my Line into Mind Maps - Live from #CAST2015
Today the rubber meets the road. We are going live today with CAST 2015, breakfast has happened, the setup for all of the bags and badges has happened, the tables are manned, the food is being served, and the prep work for the webCAST is happening now (which I do not have any real involvement with other than to be grateful that it is happening and that Ben, Dee and Paul are making it happen).
Monday at CAST is tutorial day, and since the participants pay extra for the content, I have traditionally not done a play by play of these sessions, but I will talk a bit about why I chose to be in the one I am in and what my role is in being here. I'll also talk a bit of my own observations around the topic without repeating Sekar's presentation. Also, to be kind to my readers, I will split these posts up into different posts, since many have said it's hard to follow a big long post throughout the day.
First off, each of us who are on the board and attending a tutorial are doing so as the room helper or facilitator for the tutorial. One of the things I asked all of us on the board who wanted to participate to do was to work the back channels for the event. Since there was a limited number of seats for each tutorial, we wanted to make sure that the participants in the conference go the first chance to be there. We also made sure that each of us picked a different tutorial to be part of so that we would be able to evaluate the individual sessions and be able to report back from each of them as to their effectiveness, and things we could learn to help the next year's organizers with choosing and developing solid sessions.
There are four sessions being offered this year for tutorials:
Christin Wiedemann is leading a tutorial called "Follow your Nose Testing - Questioning Rules and Overturning Convention
Dhanasekar Subramanian is leading a tutorial called "Mobile App Coverage Using Mind Maps"
Robert Sabouring is leading a tutorial called "Testing Fundamentals for Experienced Testers"
Fiona Charles is leading a tutorial called "Speaking Truth to Power: Delivering Difficult Messages"
Since we all opted to spread ourselves around the tutorial choices to be the room helpers and facilitators, I chose to work with Sekar and be part of the "Mobile App Coverage Using Mind Maps" tutorial. One of the reasons I chose to focus on this tutorial was that I have seen a variety of mindmaps used by people over the years, and I tend to focus on a fairly simplistic use of them. For me, I tend to use a core concept, and branch a few ideas off the core, and then break them down to a few words in the branches. If I need more depth, rather than make complex mind maps, I will usually just create a new mind map with another concept. the idea of having multiple branches on the same map just feels messy to me, but at the same time, having to jump to multiple maps is also messy.
Sekar's tutorial is covering two concepts at the same time. The first is giving participants a chance to work with mind maps who may not have done so in the past. The second part is testing with mobile apps and categorizing the details of the app. the benefit to using Mind Maps in the process of testing is less the rigid use of the tool and more of idea that each core concept can have several points where we can branch off.
One of the fun things we do in these tutorials is get everyone on the same page and playing with the discussions. In this tutorial we are encouraged to break into groups and discuss a variety of topics. For mine I chose music, and what I find interesting with mind maps is not so much what we add to the mind map, but why we add them. My map for music was broken into Instruments, Songs, Genres and Modes. Why did I choose those? Possibly because I am a musician, and those are the things I think about. My guess is that a casual listener might not even know that modes exist, so they wouldn't include it as part of their breakdown.
It's time for our morning break, so I'm going to call and end to this post. I'll be back with another one in a bit. Thanks for joining me today :).
Monday at CAST is tutorial day, and since the participants pay extra for the content, I have traditionally not done a play by play of these sessions, but I will talk a bit about why I chose to be in the one I am in and what my role is in being here. I'll also talk a bit of my own observations around the topic without repeating Sekar's presentation. Also, to be kind to my readers, I will split these posts up into different posts, since many have said it's hard to follow a big long post throughout the day.
First off, each of us who are on the board and attending a tutorial are doing so as the room helper or facilitator for the tutorial. One of the things I asked all of us on the board who wanted to participate to do was to work the back channels for the event. Since there was a limited number of seats for each tutorial, we wanted to make sure that the participants in the conference go the first chance to be there. We also made sure that each of us picked a different tutorial to be part of so that we would be able to evaluate the individual sessions and be able to report back from each of them as to their effectiveness, and things we could learn to help the next year's organizers with choosing and developing solid sessions.
There are four sessions being offered this year for tutorials:
Christin Wiedemann is leading a tutorial called "Follow your Nose Testing - Questioning Rules and Overturning Convention
Dhanasekar Subramanian is leading a tutorial called "Mobile App Coverage Using Mind Maps"
Robert Sabouring is leading a tutorial called "Testing Fundamentals for Experienced Testers"
Fiona Charles is leading a tutorial called "Speaking Truth to Power: Delivering Difficult Messages"
Since we all opted to spread ourselves around the tutorial choices to be the room helpers and facilitators, I chose to work with Sekar and be part of the "Mobile App Coverage Using Mind Maps" tutorial. One of the reasons I chose to focus on this tutorial was that I have seen a variety of mindmaps used by people over the years, and I tend to focus on a fairly simplistic use of them. For me, I tend to use a core concept, and branch a few ideas off the core, and then break them down to a few words in the branches. If I need more depth, rather than make complex mind maps, I will usually just create a new mind map with another concept. the idea of having multiple branches on the same map just feels messy to me, but at the same time, having to jump to multiple maps is also messy.
Sekar's tutorial is covering two concepts at the same time. The first is giving participants a chance to work with mind maps who may not have done so in the past. The second part is testing with mobile apps and categorizing the details of the app. the benefit to using Mind Maps in the process of testing is less the rigid use of the tool and more of idea that each core concept can have several points where we can branch off.
One of the fun things we do in these tutorials is get everyone on the same page and playing with the discussions. In this tutorial we are encouraged to break into groups and discuss a variety of topics. For mine I chose music, and what I find interesting with mind maps is not so much what we add to the mind map, but why we add them. My map for music was broken into Instruments, Songs, Genres and Modes. Why did I choose those? Possibly because I am a musician, and those are the things I think about. My guess is that a casual listener might not even know that modes exist, so they wouldn't include it as part of their breakdown.
It's time for our morning break, so I'm going to call and end to this post. I'll be back with another one in a bit. Thanks for joining me today :).
Saturday, August 1, 2015
Testing the IoT - Live from #TestRetreat
Ah yes, The Internet of Things! that weird cross section of devices and services that are super specific or are focused on areas that are still being defined. We hear about thermostats that learn, or devices that report blood sugar levels for people with diabetes.
The Internet of things are going to introduce all sorts of devices and needs for testing that come in all different shapes or sizes. Scott Allman brought in a bunch of interesting devices that are tiny and can be deployed for simple purposes. Imagine a device the size of a nine volt battery that can act as a full stack web server.
One of the fun aspects about these tiny devices is that they often have very primitive interfaces, or raw Linux style interfaces. This encourages people to play around with the devices and interact with the device. Additionally, with these tiny devices, it's possible to create multiple systems that can be deployed very cheaply, so we can set up little networks and sandboxes for very little cost.
Some of the challenges we face with these devices are in the realm of security, as well as the way that the system is powered (battery vs. dedicated power, using protocols like low energy bluetooth, etc.). there are also devices that use accelerometers, so part of testing includes subjecting the device to motion. Additional tests we might care about would be related to power, heat, durability, responsiveness, performance, usability, and accessibility. Many of the test approaches and themes that we use for mobile testing will be able to be applied to the Internet of Things as well.
What I find interesting is that these could be very low priced devices to implement and test with performance and security tools. I could see being able to create a small network of devices and implement jMeter and Kali Linux to poke around with. I joked that it would be fun to be able to have a belt where each section is a separate server, and HSA and load balancing applications could be experimented with.
I noticed that in this talk, I kept thinking about using these devices for servers and applications I'm already familiar with. There's lots of applications I haven't even remotely considered, and I know that with time as I get more familiar with uses, I can start applying ideas for these devices and see what other avenues of exploration I can discover.
The Internet of things are going to introduce all sorts of devices and needs for testing that come in all different shapes or sizes. Scott Allman brought in a bunch of interesting devices that are tiny and can be deployed for simple purposes. Imagine a device the size of a nine volt battery that can act as a full stack web server.
One of the fun aspects about these tiny devices is that they often have very primitive interfaces, or raw Linux style interfaces. This encourages people to play around with the devices and interact with the device. Additionally, with these tiny devices, it's possible to create multiple systems that can be deployed very cheaply, so we can set up little networks and sandboxes for very little cost.
Some of the challenges we face with these devices are in the realm of security, as well as the way that the system is powered (battery vs. dedicated power, using protocols like low energy bluetooth, etc.). there are also devices that use accelerometers, so part of testing includes subjecting the device to motion. Additional tests we might care about would be related to power, heat, durability, responsiveness, performance, usability, and accessibility. Many of the test approaches and themes that we use for mobile testing will be able to be applied to the Internet of Things as well.
What I find interesting is that these could be very low priced devices to implement and test with performance and security tools. I could see being able to create a small network of devices and implement jMeter and Kali Linux to poke around with. I joked that it would be fun to be able to have a belt where each section is a separate server, and HSA and load balancing applications could be experimented with.
I noticed that in this talk, I kept thinking about using these devices for servers and applications I'm already familiar with. There's lots of applications I haven't even remotely considered, and I know that with time as I get more familiar with uses, I can start applying ideas for these devices and see what other avenues of exploration I can discover.
Teaching Testing to Programmers - Live from #TestRetreat
We hear a lot about teaching software testers how to program, but do we hear the converse? Why don't we ever hear about software testers teaching programmers how to test? Jesse Alford thinks this deserves to be a two way street and that there's a lot of opportunity to have engineers pick up exploratory testing skills to support the team in growing and learning.
One thing that should be mentioned is that Jessie works with Pivotal Labs, and Pivotal is a pretty awesome team with a lot of high functioning people on their team, and also have a fairly good number of engineers who are well versed in software testing methodologies.
There are a number of games that we use as testers that are well known. We use The Dice Game, we use The Pyramid Game, we use James Lyndsey's Black Box Puzzles, and we use simple games like "The Pen Test" that can be infuriating, but can teach us how we model problems and apply testing principles to those games.
A phrase that Scott Allman shared, and I like a great deal, is that "our job as testers is to discover laws that have been smuggled into the model universe that the programmers didn't know about". Having the opportunity to share some of these exercises allows us to share this mindset with programmers in an area that is separate from their core experience.
One suggestion that Jessie encourages is to have two people be part of the game, especially when the game in question has a "find the rule" element to it (Dice Game, Zendo, Pen Test, etc.). One person runs the game, i.e. they know the rule and the aspect that makes the game relevant. Another participant is there to help coach the person playing the game, but they don't know what the game rule is. they are familiar with the game itself, but they don't specifically know what the particular rule for that game is. that way, the coach is also operating from a level of not knowing the specific answer, but still being able to coach the participant in how they might go about solving the problem. One point that Jesse made that makes the games themselves at times less effective is the "all knowing" game master. The coaching the game master provides turns into maddening hints, coming from the person who knows what the game is. By having a third person who doesn't know the rule, the feedback on trying to solve the problem is less biased and less of that "knowing glance".
Another way that I personally would encourage having a programmer get involved with testing is to have them participate in a few Weekend Testing sessions. These are great training grounds for session based testing, as well as to explore some interesting avenues for software testing via individual topics. The cool thing about doing these sessions is that a lot of them are focused on black box skills, and they allow everyone to discuss what they are working on and learning in the process.
It's fun to take the chance to learn from each other, and testing games are an easy way to get programmers to see what it is we do and how we do it.
One thing that should be mentioned is that Jessie works with Pivotal Labs, and Pivotal is a pretty awesome team with a lot of high functioning people on their team, and also have a fairly good number of engineers who are well versed in software testing methodologies.
There are a number of games that we use as testers that are well known. We use The Dice Game, we use The Pyramid Game, we use James Lyndsey's Black Box Puzzles, and we use simple games like "The Pen Test" that can be infuriating, but can teach us how we model problems and apply testing principles to those games.
A phrase that Scott Allman shared, and I like a great deal, is that "our job as testers is to discover laws that have been smuggled into the model universe that the programmers didn't know about". Having the opportunity to share some of these exercises allows us to share this mindset with programmers in an area that is separate from their core experience.
One suggestion that Jessie encourages is to have two people be part of the game, especially when the game in question has a "find the rule" element to it (Dice Game, Zendo, Pen Test, etc.). One person runs the game, i.e. they know the rule and the aspect that makes the game relevant. Another participant is there to help coach the person playing the game, but they don't know what the game rule is. they are familiar with the game itself, but they don't specifically know what the particular rule for that game is. that way, the coach is also operating from a level of not knowing the specific answer, but still being able to coach the participant in how they might go about solving the problem. One point that Jesse made that makes the games themselves at times less effective is the "all knowing" game master. The coaching the game master provides turns into maddening hints, coming from the person who knows what the game is. By having a third person who doesn't know the rule, the feedback on trying to solve the problem is less biased and less of that "knowing glance".
Another way that I personally would encourage having a programmer get involved with testing is to have them participate in a few Weekend Testing sessions. These are great training grounds for session based testing, as well as to explore some interesting avenues for software testing via individual topics. The cool thing about doing these sessions is that a lot of them are focused on black box skills, and they allow everyone to discuss what they are working on and learning in the process.
It's fun to take the chance to learn from each other, and testing games are an easy way to get programmers to see what it is we do and how we do it.
Bringing Energy Back to Testing - Live from #TestRetreat
Picture the scenario. A tester has been in the game for a number of years. They know the details, they know the product, they've done countless stories, and at some point, the testing becomes rote and paint by numbers. The thrill is gone. the excitement level has left the building. Do you see yourself in this scenario? If not, fabulous, this talk is not for you ;).
If, however, you have been in this situation, or currently see yourself in this situation, then Phil McNeely's session on "Bringing Energy Back to Testing" is for you. I know how this feels. I've been there and done that. At a certain point, I lost the joy and fun that testing used to provide, and at some point, I was just going through the motions. It wasn't intentional on my part, I didn't intend to go on autopilot, but I did find that there have been times where I just did what I had to do, and often what I really didn't want to do. Often the reasons came down to doing the same thing over and over. Sometimes people just burn out. For many, their passion is somewhere else, like snowboarding, or knitting, or writing a novel. Fort those people, it helps a lot to accept that their true passions lie elsewhere and to encourage them to invest in those areas to the point they feel they are getting satisfaction there, and the follow on effect is that they can focus on what they do at work.
In my case, I often found myself overcommitted to too many good things. It's not that I was necessarily burned out on my work, but that I was committed to my day job, and to writing, and to teaching, and to community engagement. In short, I found I was spreading myself too thin in too many areas. At those times, it became apparent that I need to "give myself a haircut" across the board. When a tester who was at one time productive seems to be less focused or engaged, it might be worth making a lunch date and just getting to see what is going on in that tester's life at that moment, both inside and outside of work. It's possible you might discover that they have recently become the PTA president at their school, or they have taken on an important but time consuming position at their church, or there may be a family situation due to an illness or situation in the family that is drawing upon their energy. I know when I have too many things happening in my life, every area suffers, and yes, that includes work. By realizing that energy commitments have changed, we can help that person (or ourselves) consider what options we have to make modifications.
I wrote about this a few days ago over at the ITKE Uncharted Waters blog, but something I am using to help me stave off this challenge is using an Objective Journal. By having me consider what I am working on, and questioning it on a regular basis, I can keep myself engaged with the problem, rather than waiting for something to complete and going of and working on something else that's not on target or, sometimes, not even remotely productive. The trick for me with the Objective Journal is that it allows me to see small, everyday wins. By seeing those wins, I stay motivated and excited.
Ultimately, regardless of how engaged or not engaged we are, at the end of the day we need to realize that WE are the ones that need to develop our motivation. We can encourage and offer to help others achieve motivation, but externally motivated people tend to not stay motivated for too long. Internally motivated people can stoke that fire indefinitely, so work to encourage that spark in others, but more importantly, help them develop and maintain that spark in themselves.
If, however, you have been in this situation, or currently see yourself in this situation, then Phil McNeely's session on "Bringing Energy Back to Testing" is for you. I know how this feels. I've been there and done that. At a certain point, I lost the joy and fun that testing used to provide, and at some point, I was just going through the motions. It wasn't intentional on my part, I didn't intend to go on autopilot, but I did find that there have been times where I just did what I had to do, and often what I really didn't want to do. Often the reasons came down to doing the same thing over and over. Sometimes people just burn out. For many, their passion is somewhere else, like snowboarding, or knitting, or writing a novel. Fort those people, it helps a lot to accept that their true passions lie elsewhere and to encourage them to invest in those areas to the point they feel they are getting satisfaction there, and the follow on effect is that they can focus on what they do at work.
In my case, I often found myself overcommitted to too many good things. It's not that I was necessarily burned out on my work, but that I was committed to my day job, and to writing, and to teaching, and to community engagement. In short, I found I was spreading myself too thin in too many areas. At those times, it became apparent that I need to "give myself a haircut" across the board. When a tester who was at one time productive seems to be less focused or engaged, it might be worth making a lunch date and just getting to see what is going on in that tester's life at that moment, both inside and outside of work. It's possible you might discover that they have recently become the PTA president at their school, or they have taken on an important but time consuming position at their church, or there may be a family situation due to an illness or situation in the family that is drawing upon their energy. I know when I have too many things happening in my life, every area suffers, and yes, that includes work. By realizing that energy commitments have changed, we can help that person (or ourselves) consider what options we have to make modifications.
I wrote about this a few days ago over at the ITKE Uncharted Waters blog, but something I am using to help me stave off this challenge is using an Objective Journal. By having me consider what I am working on, and questioning it on a regular basis, I can keep myself engaged with the problem, rather than waiting for something to complete and going of and working on something else that's not on target or, sometimes, not even remotely productive. The trick for me with the Objective Journal is that it allows me to see small, everyday wins. By seeing those wins, I stay motivated and excited.
Ultimately, regardless of how engaged or not engaged we are, at the end of the day we need to realize that WE are the ones that need to develop our motivation. We can encourage and offer to help others achieve motivation, but externally motivated people tend to not stay motivated for too long. Internally motivated people can stoke that fire indefinitely, so work to encourage that spark in others, but more importantly, help them develop and maintain that spark in themselves.
What is the Career Path for a Tester? - Live from #TestRetreat
Carl Shaulis asked a simple question, or what seemed to be a simple question... what are some of the skills needed at various points of a career for a software tester?
There are many variations of software testing and approaches to software testing, but for many of us, it seems that there is a specific path. The first round is what we as a group called "the muscle", i.e. classic manual software testing. The starting point for this is, to borrow from Jon Bach, a need for curiosity. Jon has said that he can teach people technical stuff as needed, but he can't teach people how to be curious (I will come back and get the quote and post where he says this, but for now, forgive the live blogger and lack of immediate attribution ;) ).
Another aspect we discussed is that the skill levels are not specifically better at higher job levels. Many of the skills of a Level 1 tester are still applied at higher levels. they don't necessarily have specialized knowledge, but they do have experience using it over several years. What is expected for any level is the ability to evaluate a product, to look at the product with an eye to look at a workflow and determine if something is out of place or not working in the way that people expect. Critical thinking skills are valuable at any level of the job. The differentiators tend to not be the skills, but the level of influence within the organization the individual has. Junior testers and senior testers are often differentiated not so much by their skill level, but with experience and leadership, as well as overall influence.
One obvious question to ask is "does a tester need to learn how to code and make coding part of their job if they want to advance?" My answer is that, if you want to be a toolsmith and work on test tooling, then yes, programming is essential. If you don't aim to be a toolsmith, or if you are not interested in focusing on automation or tooling, then programming may not be essential. Having said that, I think that many more questions are capable of being asked and evaluated when a tester can look at the underlying code and understand what is happening, even if just in a general sense.
Much of these discussions come into play so that they can have a shorthand when it comes to job titles, compensation, and ability to allocate people into an organization. I've primarily worked in smaller companies the past fifteen years, but during the first ten years of my career, I worked with Cisco Systems, which went from a smallish 300 person company when I joined it in 1991 to a 50,000 plus person company when I left in 2001. Early in the life of Cisco, job titles were less important than they were later. When dealing with a company that is much larger, titles and the shapes of the "cogs" starts to matter. Within smaller teams, generalists and people able to cover lots of different areas are much more important, and there's a fluidity in the work that we do. Career path is less relevant in a smaller company, and the reward aspects are different. In many ways, in a smaller company, you are not rewarded with titles or advancement, you are rewarded with influence (and in some cases, with money or equity).
As a closer, it was suggested that we check out the Kitchen Soap article "On Being a Senior Engineer". There is a lot of meat in this one so I might do a follow-on post just on this article :). ETA: Coming back to this two and a half years later, I was asked if I would consider adding "7 Reasons You Can't Get a Junior Web Developer Job" as a follow-on read. Granted, a little outside of the software testing space, but many of the same issues also fit this discussion, so yes, considered and added :).
There are many variations of software testing and approaches to software testing, but for many of us, it seems that there is a specific path. The first round is what we as a group called "the muscle", i.e. classic manual software testing. The starting point for this is, to borrow from Jon Bach, a need for curiosity. Jon has said that he can teach people technical stuff as needed, but he can't teach people how to be curious (I will come back and get the quote and post where he says this, but for now, forgive the live blogger and lack of immediate attribution ;) ).
Another aspect we discussed is that the skill levels are not specifically better at higher job levels. Many of the skills of a Level 1 tester are still applied at higher levels. they don't necessarily have specialized knowledge, but they do have experience using it over several years. What is expected for any level is the ability to evaluate a product, to look at the product with an eye to look at a workflow and determine if something is out of place or not working in the way that people expect. Critical thinking skills are valuable at any level of the job. The differentiators tend to not be the skills, but the level of influence within the organization the individual has. Junior testers and senior testers are often differentiated not so much by their skill level, but with experience and leadership, as well as overall influence.
One obvious question to ask is "does a tester need to learn how to code and make coding part of their job if they want to advance?" My answer is that, if you want to be a toolsmith and work on test tooling, then yes, programming is essential. If you don't aim to be a toolsmith, or if you are not interested in focusing on automation or tooling, then programming may not be essential. Having said that, I think that many more questions are capable of being asked and evaluated when a tester can look at the underlying code and understand what is happening, even if just in a general sense.
Much of these discussions come into play so that they can have a shorthand when it comes to job titles, compensation, and ability to allocate people into an organization. I've primarily worked in smaller companies the past fifteen years, but during the first ten years of my career, I worked with Cisco Systems, which went from a smallish 300 person company when I joined it in 1991 to a 50,000 plus person company when I left in 2001. Early in the life of Cisco, job titles were less important than they were later. When dealing with a company that is much larger, titles and the shapes of the "cogs" starts to matter. Within smaller teams, generalists and people able to cover lots of different areas are much more important, and there's a fluidity in the work that we do. Career path is less relevant in a smaller company, and the reward aspects are different. In many ways, in a smaller company, you are not rewarded with titles or advancement, you are rewarded with influence (and in some cases, with money or equity).
As a closer, it was suggested that we check out the Kitchen Soap article "On Being a Senior Engineer". There is a lot of meat in this one so I might do a follow-on post just on this article :). ETA: Coming back to this two and a half years later, I was asked if I would consider adding "7 Reasons You Can't Get a Junior Web Developer Job" as a follow-on read. Granted, a little outside of the software testing space, but many of the same issues also fit this discussion, so yes, considered and added :).
How Can We Interview Testers Better? - Live from #TestRetreat
One of the challenges that anyone who has been involved with hiring software testers can tell you is that it can be maddening to interview testers. We can find people, but getting the right people is often a struggle. We have all had the experience of reading a resume and seeing what looks to be very promising history and experience, only to have them in an interview and have to ferret out what they actually understand or do not understand. is there a way that we can do this better? Dwayne Green led a group of us to discuss how we approach these interviews and how we can improve the process.
Many of us have worked through resumes and had to make phone screens or initial interviews, and a common phrase that came up was to "audition" the candidate. There are several approaches to auditions that we can use. Some people will use a sample program and walk through how to test it. Some people like to use their own company's product as part of the audition, and to see how the candidate tests the product, or if they can find problems we already know about.
A discussion I recently had with our VP of Engineering at my company was the way that we expect people to work. Most of the programmers that write code have certain things open on their desk at all times: their IDE or programming environment, a browser with a google tab open and Stack Overflow or some other reference sites. the point being made is that, when we work, we tend to have these tools open to help us get to the things we need. When we interview, we deprive these candidates of that ability, and we expect them to "code" on a whiteboard. One of the ways that was suggested that we could change this would be to ask in advance what they like to use to work, and to feel free to bring that in with them. When we put the audition challenge in front of them, we also say "use the tools you would use". This has tended to give a more representative view as to how those programmers actually code. We discussed the idea that people should do the same thing when it comes to software testing. Let a candidate come in and use the tools they already understand.
One of the questions that we asked was "how could we make these interview approaches more real, but avoid a situation where we are making people "work for free" as part of the interview". There is always a danger that putting people into a simulation or scenario with the company's product runs the risk of replicating real work without compensation. One approach could be to use a virtual machine with a version of a release and a story that has already been worked, and have a list of issues that have already been discovered. Do we want to have them run an abbreviated test session and see what they discover from our list? Do we want to have them sit with our team members and pair for a particular period of time? In my view, I think we need to make clear that we are going over material that has already been covered, and that we are using it as an evaluation criteria and not as a way to get "free testing" out of someone.
Some people like to use games or use sample programs to do these experiments. The issue there is that some people are not good at games, or are not good at particular applications. Does that mean they are a bad tester? Probably not, but it does mean we need to take into consideration multiple avenues. They might not be good at one area, but be great in others. If we find that they don't do well in multiple areas, that might well give insights as to how well they will perform or not perform in our work environment.
We all discussed some of the worst experiences we had with interviews, and I shared that would often be asked some strange questions related to math problems or other specific questions that seemed to have no bearing on what we would be testing. Over time, I came to realize that these questions served one purpose: does this person think like I do? To be fair, it's been years since I've had to deal with those types of questions, but I know that some people still do, and if I can make one plea, it's to say "knock it off!" ;).
Many of us have worked through resumes and had to make phone screens or initial interviews, and a common phrase that came up was to "audition" the candidate. There are several approaches to auditions that we can use. Some people will use a sample program and walk through how to test it. Some people like to use their own company's product as part of the audition, and to see how the candidate tests the product, or if they can find problems we already know about.
A discussion I recently had with our VP of Engineering at my company was the way that we expect people to work. Most of the programmers that write code have certain things open on their desk at all times: their IDE or programming environment, a browser with a google tab open and Stack Overflow or some other reference sites. the point being made is that, when we work, we tend to have these tools open to help us get to the things we need. When we interview, we deprive these candidates of that ability, and we expect them to "code" on a whiteboard. One of the ways that was suggested that we could change this would be to ask in advance what they like to use to work, and to feel free to bring that in with them. When we put the audition challenge in front of them, we also say "use the tools you would use". This has tended to give a more representative view as to how those programmers actually code. We discussed the idea that people should do the same thing when it comes to software testing. Let a candidate come in and use the tools they already understand.
One of the questions that we asked was "how could we make these interview approaches more real, but avoid a situation where we are making people "work for free" as part of the interview". There is always a danger that putting people into a simulation or scenario with the company's product runs the risk of replicating real work without compensation. One approach could be to use a virtual machine with a version of a release and a story that has already been worked, and have a list of issues that have already been discovered. Do we want to have them run an abbreviated test session and see what they discover from our list? Do we want to have them sit with our team members and pair for a particular period of time? In my view, I think we need to make clear that we are going over material that has already been covered, and that we are using it as an evaluation criteria and not as a way to get "free testing" out of someone.
Some people like to use games or use sample programs to do these experiments. The issue there is that some people are not good at games, or are not good at particular applications. Does that mean they are a bad tester? Probably not, but it does mean we need to take into consideration multiple avenues. They might not be good at one area, but be great in others. If we find that they don't do well in multiple areas, that might well give insights as to how well they will perform or not perform in our work environment.
We all discussed some of the worst experiences we had with interviews, and I shared that would often be asked some strange questions related to math problems or other specific questions that seemed to have no bearing on what we would be testing. Over time, I came to realize that these questions served one purpose: does this person think like I do? To be fair, it's been years since I've had to deal with those types of questions, but I know that some people still do, and if I can make one plea, it's to say "knock it off!" ;).
On Being The "Least Talented" Person in the Room - Live from #TestRetreat
Good morning from Grand Rapids, Michigan. I'm still functioning on West Coast time, but everyone else around me thinks it's the middle/late morning, so I need to kick in and get with the groove of TestRetreat. For those not familiar, TestRetreat has been an ongoing event the past few years hosted by Matt Heusser and is an un-conference event with a broad list of ideas and talks being presented by people who have interests in given areas.
A piece of housekeeping this time around. In the past, I have Live Blogged events in a day long format and have updated each page during the day. Due to the way that CAST is going to be run this year, and with the way that I will be more facilitating rather than just attending sessions, I won't be able to do the live blog the same way. Instead of doing a running log, I'll be making individual posts for each session or event. Downside here is that there will be multiple posts in a given day. Let me know which way you find most useful :).
Out first session is being hosted by Ajay Balamurugadas, and he's asking about the interesting dichotomy in that he at times feels like he is the least talented person in the room, yet he is the one that gets invited to speak at and participate in conferences. Why is that?
I've personally struggled with this attitude myself, and in part, I think the title is a little misleading, but the overall perception is an important one. By starting with the idea that we are the least talented person in the room, we are stating that we feel that we have a lot we can learn, and we are willing to put in the time to learn it. I would be hard pressed to consider Ajay the least talented person in any room, but I would say that that sense of humility and desire to learn as much as possible drives him and gives him the motivation to keep learning and growing. Adding to this is the fact that when you do create a reputation as a person who is actively engaged in the broader community, you can be held to a standard that is unfair to where you actually are. As I've talked to people over the years, I've likewise seen that they consider me to be a "thought leader" or "testing guru", and at times, I've considered their expectations to be unfounded. It's not that I have any super level of expertise, but the fact that I've written about or talked about the problem in any capacity tends to make people think I know or understand more than I do.
I'm not sure there is an easy way to deal with this other than to acknowledge that there is a reputation that being actively engaged gives someone. When we say "just because I write and talk about these topics, don't think that I am somehow superior to those who don't write or speak on topics" (fact is, some people who are mega performers do not engage in the broader community, and that's totally OK). I personally talk to my team with the idea that I am allowed one completely stupid question of each member of my team per day, and that they should accept it. To date, I've only had a couple of instances where people actually said "wow, you should know this". Many times, the answer I get is "no, that's not a dumb question. We don't really describe or define that area very well." the one thing I think is important is that the onus is on us to communicate both what we understand and what we don't understand. Also, it's important to get a gauge as to the expectations of what people want to see from us, and that we are meeting those expectations. If we are not, we should determine what their expectation actually are.
A piece of housekeeping this time around. In the past, I have Live Blogged events in a day long format and have updated each page during the day. Due to the way that CAST is going to be run this year, and with the way that I will be more facilitating rather than just attending sessions, I won't be able to do the live blog the same way. Instead of doing a running log, I'll be making individual posts for each session or event. Downside here is that there will be multiple posts in a given day. Let me know which way you find most useful :).
Out first session is being hosted by Ajay Balamurugadas, and he's asking about the interesting dichotomy in that he at times feels like he is the least talented person in the room, yet he is the one that gets invited to speak at and participate in conferences. Why is that?
I've personally struggled with this attitude myself, and in part, I think the title is a little misleading, but the overall perception is an important one. By starting with the idea that we are the least talented person in the room, we are stating that we feel that we have a lot we can learn, and we are willing to put in the time to learn it. I would be hard pressed to consider Ajay the least talented person in any room, but I would say that that sense of humility and desire to learn as much as possible drives him and gives him the motivation to keep learning and growing. Adding to this is the fact that when you do create a reputation as a person who is actively engaged in the broader community, you can be held to a standard that is unfair to where you actually are. As I've talked to people over the years, I've likewise seen that they consider me to be a "thought leader" or "testing guru", and at times, I've considered their expectations to be unfounded. It's not that I have any super level of expertise, but the fact that I've written about or talked about the problem in any capacity tends to make people think I know or understand more than I do.
I'm not sure there is an easy way to deal with this other than to acknowledge that there is a reputation that being actively engaged gives someone. When we say "just because I write and talk about these topics, don't think that I am somehow superior to those who don't write or speak on topics" (fact is, some people who are mega performers do not engage in the broader community, and that's totally OK). I personally talk to my team with the idea that I am allowed one completely stupid question of each member of my team per day, and that they should accept it. To date, I've only had a couple of instances where people actually said "wow, you should know this". Many times, the answer I get is "no, that's not a dumb question. We don't really describe or define that area very well." the one thing I think is important is that the onus is on us to communicate both what we understand and what we don't understand. Also, it's important to get a gauge as to the expectations of what people want to see from us, and that we are meeting those expectations. If we are not, we should determine what their expectation actually are.
Subscribe to:
Posts (Atom)