Thursday, February 28, 2013

Selenium SF Meetup: Tales From the Selenium Testing Trenches

For those who haven't seen it, and are in the SF Bay Area today, the San Francisco Selenum Meetup group will be getting together tonight. I'm setting the stage with this post, and I'll update it later this evening when the event starts so that those who can't be there can get a taste of what's happening.

So a static blog to set up a live blog for later... yeah, that's it :).

With that out of the way... Thursday, February 28, 2013 at 6:30 p.m. (we walked in at 7:15 p.m. and tamales were never so appealing, lot of traffic on 101 tonight, my thanks to my Quality Director for the ride up :) ):

Tales From the Selenium Testing Trenches

Lookout Mobile Security
One Front St, Ste 2700,
San Francisco, CA
6:30 p.m. - 9:00 p.m.

The first speaker is Giri Nandigam of TRUSTe about their Tracker Detection Tool and how Selenium has become part of their mix to test, develop and move forward. TRUSTe is in the business of protecting customer data, and to do that, they have to make sure that large amounts of data can be safeguarded, and that involves dealing with some huge systems. Their goal was to try to comb networks to look for data and verify that it is not being exposed. Their challenges that they were facing were having issues where JavaScript engines would throw too many errors, and trying to get systems that could be maintained in a realistic manner. Their idea was to use Selenium as a crawler, and to allow them to use Selenium to go from page to page, to collect all of the links (thousands of them) and then export out those links as HAR files for later access. Also, they've made a browser hack so that they can get access to the cookie store for requests (an interesting way to go about doing some protective data mining, to be sure).

These kind of talks are always interesting because we get to see what other organizations do with Selenium, and while we are used to thinking of Selenium and its framework to "test", it's a system for computer aided processing and step automation, and looking to see what data can be gathered.

Taking a little break, and nature calls, so I'll be back in a bit. Oh, and Ken got the last Tamale, if any are here and looking for more food ;).


The second speaker is Dr. Wenhua Wang of Marin Software, who will be discussion ways to use Selenium to test IE. Due to a lot of multi-browser needs ,and yes, a large customer that mandates IE7 as part of the support deal, IE is not going to be something I will be seeing less of any time soon. Marin has   lot of legacy code and the idea of creating new automation tests was and still is a daunting situation to deal with.

Also, many of their tests used XPath locators (sometimes very helpful, but often very expensive if not implemented in the right scope... hey, I actually understand what that last sentence means now ;) ). They decided that, rather tan start from scratch, they would see how much of their existing test code they could refactor, especially with the gains that WebDriver now gives to allowing an easy retrofitting of Selenium drivers, and instead of changing the testing code itself, use the time and energy to update and refresh the locators and methods to access them.

The big win and goal of these changes was to try to explore techniques that would minimize the code maintenance, plus allow for a maximum on the return of effort with the lowest overall cost. Developing automation tests with well supported techniques will help make sure that the churn of test change is minimal.

One interesting question from the audience was to ask "if you could do this all with no budgetary impact and all the resources necessary, what would you do differently"? The best answer came from Ken, who was sitting right next to me... "tell your developers to make sure that very element has an id or a class!" Frankly, I couldn't agree more!

I said I was looking forward to seeing Chris McMahon and Zeljko Filipin being in town for this, and sure enough, they came. I felt bad that I didn't have any time this week to get out and hang with them while they were in town, but this in a small way makes up for that. My thanks also to Ken for being my ride to the city tonight, and letting me take a much needed nap on the way up (that's a topic for another post, but not just yet ;) ). To all of my Selenium compatriots, good to se you again, and we get a treat in two weeks, another session at Pivotal Labs, this time to celebrate Elizabeth Hendrickson's in-print launch of Explore It. That will be on Tuesday, March 12, 2013, and I hope to see you there :).

Wednesday, February 27, 2013

TESTHEAD Giveaway: WIN a FREE Selenium Book!!!

And now for something "Completely Different"... and yet, not.

For those who have been following this blog for the past month, I've been working through "Selenium 2 Testing Tools" and doing a long form review of the exercises and materials. I also, last month, did a review of another book, "Selenium Testing Tools Cookbook", which is also available from the publisher of Selenium 2 Testing Tools, i.e. Packt Publishing in Birmingham, UK.

So why am I mentioning this? Because Packt Publishing is partnering with TESTHEAD to give away three FREE copies of the Selenium Testing Tools Cookbook!

OK, do I have your attention :)?

Here's some of the details about "Selenium Testing Tools Cookbook" from the Packt Publishing site:

  • Learn to leverage the power of Selenium WebDriver with simple examples that illustrate real world problems and their workarounds.
  • Each sample demonstrates key concepts allowing you to advance your knowledge of Selenium WebDriver in a practical and incremental way.
  • Explains testing of mobile web applications with Selenium Drivers for platforms such as iOS and Android
  • Uses a variety of examples and languages such as Java, Ruby and C#.

Read more about this book and download a free Sample Chapter

How to Enter

All you need to do is head on over to the book page (Selenium Testing Tools Cookbook), look through the product description of the book, and drop a line via the comments below this post to let us know what interests you the most about the book. It’s that simple.

Winners from the U.S. and Europe can either choose a physical copy of the book or the eBook. Users from other locales are limited to the eBook only.


The contest will close on March 10, 2013. Winners will be contacted by email, so be sure to use your real email address when you comment (you can, of course write it as my (dot) name (at) emailProvider (dot) com). The winners will be chosen by a random number generator, and I'll only count people once, so no leaving a bunch of comments to boost your chances.

Please feel free to share this post with any and all who want to participate.

Tuesday, February 26, 2013

SummerQAMP 2013 Training Materials... Home Stretch!!!

Today, SummerQAmp put out a press release talking about what they are doing for 2013, and the fact that they are working with a number of organizations to "pre-train" the interns that will be participating this year. For the full press release, take a look here: 

SummerQAmp Embarks on Its Second Year to Create Internships at Leading Technology Companies in Eight U.S. Cities.

Of note is this:

"In addition, this year we are offering a more in depth online curriculum developed in collaboration with the Association for Software Testing, who is graciously providing the educational materials for students to sharpen their QA skills during the program..."
I want to say thank you to everyone that has taken the time and put forth the effort to help us develop materials and consider topics and provide details. A special thanks goes out to Smita Mishra, who single handedly (at least to date ;) ) took on the module about "What is the Software Development Process?"

Again, we are in the home stretch, but we are not out of the woods yet :). If you would like to help us get over this last hill and make our goal line on time (i.e. March 1, 2013) please head over to the EdSig Forum and weigh in with your ideas, on Bias, on Testing, on Context, on Software Development Process, or any other areas you think would be beneficial. Remember, our goal is "what did you wish you had known on your first day?" If we are not representing it, tell us, and tell us quickly :).

Monday, February 25, 2013

Podcast Review: How to Program with Java

During the process of working through the issues and question related to the PRACTICUM series I'm going through with Selenium 2 Testing Tools, I found that a refresher on my Java would be a good place to spend some time.

There are a lot of resources out there for those who want to learn about Java, and for some of us, there are different things that resonate. Personally, I like hearing a human voice, one that can coach you through areas that might not make obvious sense when you are working with them. Many podcasts start from the premise that you are already experienced, and that you are looking to hear special topics or more of a conversational podcast about the state of your chosen language. As an example, I enjoy listening to Ruby Rogues for this very reason.

For a choice to listen to and contemplate topics related to working with Java, I found "How to Program with Java" in iTunes. This podcast started in September of 2012, and roughly each week, Trevor Page puts together a podcast talking the details and the nuts and bolts about what Java is, what it does, and how to use it. The content is focused, it's easy to follow, and the coaching style helps a lot. Trevor takes the time to explain both simple things and those areas that are a little less simple to grasp. There is also a companion web site you can visit at that gives you access to other materials, as well as video tutorials and an E-book with exercises (those you pay for, but the podcast and other posts are free).

For those of you out there who want to learn a new language, and if Java is something you might see being part of your future, or you need to come back to Java after a long time away, Trevor's podcast would be a worthwhile listen.

Sunday, February 24, 2013

Ya' Gotta' Commit

Yesterday I and a friend of mine brought some of our kids up snowboarding. I have long considered Homewood on the Tahoe West Shore one of my favorite places to ride, and decided it would be fun to bring us all there. For my friend and his son this had been their first time up in a few years, and might be their only time coming up this season. 

As we were hitting the various runs, I noticed that my friend's son was only riding on his heel side edge. This is common for many snowboarders. they learn how to ride generally straight, or they learn to break on their heel-side edge or slide down on their heels on steep slopes. For many snowboarders, this is as far as they ever get. I've gone up with some riders who have ridden a few days a season, and even well into their thirties and forties, they are still only able to go straight or go heel-side. 

As is often the case when I snowboard with friends, I cannot help but morph into a teacher (it's an annoying trait, I know), but I asked him why he wasn't doing toe-side turns? He shrugged and mumbled, and I suggested, if he was interested, that I could teach him how to do toe-side turns. It's really not that hard to do, once you get the technique down. I even offered to share with him my "secret sauce" technique that has worked with everyone that has ever tried it. He moderately agreed to give it a try, and with that, I went up the hill to an open run, and gave him the following piece of advice.

When we snowboard, the most critical thing we can do is keep the center of gravity right between the balls of our feet. To do this, I asked him to visualize how he would carry a 5 gallon water bottle, full, for a distance of 100 yards if he had to. Some smart-alec's will say "throw it on my shoulder" because they want to look macho, but most of us, when pressed, will likely say that we'll carry it right in front of us, with our hips helping to support the weight. This is the answer I was after, because if he could visualize how to walk while carrying this, he would realize where his center of gravity would be. 

I then asked him to flip over and stand up on his toe side edge, and practice doing the  exercise while determining just how much edge he needed to get moving and stop. For many of the people I've shown this idea to, they have been able to pick up on it in just a few minutes. In this case, though,  while he would momentarily get it, each time I tried to get him to go further, to commit to the turn, he would panic, lose it, and crash. His frustration would take over at that point and he would get mad, or punch the now, shout "I can't do this!" or some other thing. Net result, he came dangerously close to doing it, but at the last moment, he'd bail on it, and ultimately i just had to say "well, there's the technique". You know what you need to do. Commit to it and you'll get it." 

Sometimes the most frustrating things when it comes to teaching a new skill is getting over the "commit" phase. We know that we want to do something, we can rationally understand why it would be god to do something (rationally, we understand that linking heel-side and toe-side turns gives us much better control, lets us ride a lot faster, and cover much more terrain comfortably). Still, it's one thing to say it, it's another to really be wiling to tell ourselves "I'm going to make this turn, even if by doing so I'm going to crash, hit the snow, and it's likely going to hurt". Fact is, at that point, it stops being fun, and for many people, they would rather just stick with what they know rather than go into that "punishment zone". Making the decision to commit, and go through with it, no matter how hard, no matter how uncomfortable, no matter how scary, is a real leap of faith, and a real act of courage. Often, we get to the point where we master the goal, and we then say "huh, that wasn't such a big deal!" but leading up to it, oh yes, it absolutely is. 

This need to commit, no matter the price shows up in everything. It shows up when we want to learn new testing skills. It comes up when we want to code. It comes up whenever we need to branch out into a new technical skill or gain domain knowledge of any kind. We can have all the tools and all the techniques at our disposal. Sometimes, we can just walk the steps and presto, it's all good. Sometimes, though, there's a wall we have to go over, or around, or sometimes just punch through, and at those times, tools, techniques, skills, mentors, none of that matters. Those are the moments when we are alone with our own "lizard brains" and we have to make the decision, right then and there, to commit. If we do, we will likely come out the other side better for the experience. If we don't, we can go on as we already have. Either way, the choice is always up to us, but if we want the reward... "'ya gotta' commit!" 

Saturday, February 23, 2013

Sometimes Simpler Gives a Greater Effect

As I've recently mentioned, I've been working with a friend of mine's son to teach him how to play guitar. We're very early on in the stages of this process, but already, I'm realizing that there are things that can be done to help get those "quick wins" to make it feel like you can actually make progress and keep excited.

I remember going through books for a number of years trying to make sense of what I was reading, and realizing that either the theory was over my head, or that the images being shown only told part of the story, and I'd obsess over those little bits. In truth, after I started listening to music and watching others play, I started to realize that there were a number of things that seasoned players did that were much simpler. While I was wracking my brain and my hands trying to master these intricate chords, most of the players I would see, especially in rock guitar, would usually play the chords they were hitting with just two fingers and usually just three strings. Occasionally, they would hit chords where everything would ring out, but more times than not, it was much simpler that it seemed.

I realized that most of the chords I was showing my student could also likewise be simplified a bit, and that there were two finger variations of most of the chords he could use, and they could be moved up and down the neck easily. This makes for a big win in that I can teach him real songs without the minutiae (that will, of course, come later).

We often get sidelined because we don't have enough knowledge to do everything. We get frustrated when we can't get it perfect. Let's stop thinking like that. Instead, let's see what we can do now, and let's see how effective we can be with a little bit first. Once we get our bearings, and we get accomplished, then we can expand our repertoire, but just like guitar, many of the efforts we get involved in can be performed with a lot less effort than we believe is possible. We just have to trust that a little can go farther than we think it can.

Friday, February 22, 2013

Tis The Conference Season!!!

It's almost March, when a young testers heart and mind, if they are so oriented and plugged in, starts to think about the upcoming Conference season and where they might go, who they might hear, and what they might say.

This year, I have decided to aim for three conferences. Two as a speaker, one as a supporting player, and I'll see if I can finagle one more if it makes sense and my company is cool with it.

First of all, for those in the Southern California area or those who will be attending STP-CON Spring 2013 (to be held April 22-25, 2013). I'll be there speaking on my experiences as a "Lone Tester in an Agile World". Actually, I may have to say that as an "Ex Lone Tester", as that's not my reality any longer, but I can say I certainly learned enough about what could be done, and most certainly what not to do, that it should be a fun and engaging session. My talk is on Thursday, Apr. 25, 2013, stating at 11:45 a.m.

The second conference is one I'd like to attend, and I've already made overtures as to how I can get a "crew pass" and work the show as a roving correspondent. Yes, I'm talking about the Selenium Conference 2013, to be held in Boston, MA June 10-12, 2013. My goal is to be a roving correspondent and create audio content that can then be distributed both at the conference or afterwards. The final decision still hasn't been made on this one, but I am hopeful. Either way, I'd like to see how I can be of service to this conference, and perhaps can package the materials to be available before the conference. Either way, I want to see this one success, and I want to be part of it one way or another.

I just received confirmation that I will be a full track speaker for CAST 2013, which is being held in Madison, WI, August 26-28, 2013. My topic for this conference will be how we went through and developed the (as of now) still baking content for the SummerQAmp initiative. What we learned, what went well, what went not so well, and the feedback that we have received along the way, both good and bad :).

That's what has been forecast thus far. as in all things plans may change, and I will likely look to opportunities that are weekend oriented and close to home, too. More on those opportunities as I hear about them. More to the point, I look forward to meeting as many of you as I can who can make it to any of these events :). 

PRACTICUM: Selenium 2 Testing Tools Beginner's Guide: Selenium Grid

This is the next installment (completed) of my PRACTICUM series. This particular grouping is going through David Burns' "Selenium 2 Testing Tools Beginner's Guide".

Note: PRACTICUM is a continuing series in what I refer to as a somewhat "Live Blog" format. Sections may vary in time to complete. Some may go fast. Some may take much more time to get through. Updates will be daily, they may be more frequent. Feel free to click refresh to get the latest version at any given time. If you see "End of Section" at the bottom of the post, you will know that this entry is finished :).

[Note for 2/19/2013: Compiler errors... no, that will not stand, man! I will get to the bottom of this... and I think I finally have. For those who are using Java 7 as your JDK, look towards the bottom, I discovered something that resolved our past compile problems, or at least, I think I have :). --MKL].

Chapter 8: Selenium Grid

One of the nice things I have noticed thus far with regards to the Selenium tests that I have seen is that they are relatively easy to change so that the tests can be run on multiple devices. The down side is that, so far, we have had to run them all one at a time, or switch things over to different servers, different bridges, and check them out one at a time. Also, tests are launched serially and run in order in the approaches we have taken so far. That's all cool, but is there some way that we could simplify things and just have one place we could focus our attention? The answer is "well, almost!"

Selenium Grid lets us set up a number of Selenium instances, with one central "hub" to send commands to. Selenium Grid routes the commands destined for a particular device or driver based on the configuration options and identification elements we create. While it's not an "all in one" solution, it's a nice way to go about gathering a lot of tests under one roof and varying the ability to test lots of different devices. First, though, we need to set up our environment to allow for a grid server.

Starting up Selenium Grid

Thankfully, this is an easy first step. We start up Selenium Server like we usually do, but we add a flag to it, like this:

java -jar selenium-server-standalone-2.29.0.jar -role hub

This will start up selenium server in grid mode (note, of course, use the version of selenium-server-standalone[x.xx.x].jar that you have, if you have a newer version, it should work the same).

And here's what your browser will say if you point to the server.

Cool, we've successfully started Selenium Grid. All looks good here. This is going to be the center point of our tests, and other Selenium instances will be reached from here. When we start Selenium Grid, we can see what options we have control over and which ones we can change. By passing in the port number (i.e. –port ####) We can use any value that we want (and have access to, of course).

By typing in


we can see what environments are accessible (or at least assessible as far as the grid hub is concerned).

Unlike the Selenium Grid 1.0 version, 2.0 allows a machine to host multiple servers and allow access to multiple browser drivers.

Setting up Selenium Grid with a Local Server Node

This time, we'll look at starting up Selenium Grid, as well as starting up a second Selenium server.

Enter the following command in the apropriate location (note, the previous selenium grid command needs to be running for this to work. That's not directly mentioned in the book):

Start with

java -jar selenium-server-standalone-2.29.0.jar -role hub

and then run

java -jar selenium-server-standalone..jar -role node -hub http://localhost:4444/grid/register

You should see the following in your command prompt or console:

And if all goes well, this is our Selenium Grid display:

Nice! Looks good :)!

Setting up Selenium Grid With Remote Server Nodes

While having the grid running locally is cool, to really leverage the benefits of Selenium grid, what we really want to be able to do is run Selenium Server on multiple machines, and have those machine register their connections with the main Selenium Grid hub. The more machines we have that can do this (and the different variety of machines we use for this purpose), the more we can leverage the benefits of cross-platform and parallel testing. For this purpose, here's my setup with a second Darwin device and my Windows machine:

Each of the machines needs to be registered, and each needs to point to the primary grid hub, like this:

1. Make sure the hub system is running on my main Darwin system:

 java -jar selenium-server-standalone-2.29.0.jar -role hub

2. Set up a secondary local Darwin Selenium server instance and have it pointing to the hub:

java -jar selenium-server-standalone-2.29.0.jar -role node -hub http://localhost:4444/grid/register

3. Set up the second Darwin machine to run a Selenium server and register it with the first Darwin machine's hub:

java -jar selenium-server-standalone..jar -role node -hub

4. Set up the Windows machine to run a Selenium server and register it with the first Darwin machine's hub:

java -jar selenium-server-standalone..jar -role node -hub

Cool, so if we did that all correctly, we should have a grid with three registry entries... do we?

Indeed we do. Right on! So the key here is that we have the hub instance running, and with that, we can set up as many remote environments as we have machines. For our purposes, though, I think this will be sufficient to make the point.

Setting up Selenium Grid to dedicate OS/Browser Specific Tests

If we only needed to run on one browser and one operating system, this would be a much easier situation to deal with. In that case, much of Selenium Grid would be to merely provide for parallel tests and pointing to different instances of Selenium Server, all utilizing the same browser driver. While that's probably OK in some instances, most of us have to deal with situations that require a variety of browsers. More to the point, we don't have to just consider issues of browsers (Chrome, Firefox, Opera) in their own rights, we also have to consider the challenges of running them in a variety of Operating Systems (Windows, Mac, Linux, etc.) Even with Internet Explorer being solely on Windows, there are now multiple versions and multiple OS considerations (XP, Vista, 7, 8). Safari runs on Mac, and we haven't even started talking about Mobile yet! Selenium Grid, in a manner of speaking, gives us "One Ring (Hub?) to Rule Them All!" Well, OK, perhaps not all, but there are nine options that we can leverage at this point.

We've already seen that we can set up a base hub for the Grid, and use the -hub flag to point remote instances to it. We can use the -role flag to associate remote instances of Selenium server with our Grid hub. To get the browser level into the mix, we use a flag called -browser . The following command lets up set up a Selenium Server that registers with Internet Explorer specific options:

java \
-jar selenium-server-standalone.jar \
-role node \
-hub http://localhost:4444/grid/register \
-browser browserName="internet explorer",maxInstances=1,platform=WINDOWS

If this is doing what it's supposed to do, then we will have a Grid entry that only references Windows. That's what the book shows. My reality is a different story:

Not sure if that's going to make a big difference, but suffice it to say, we're running with the parameters the book told us to run. In our later tests, we'll see if they run correctly.

Using Selenium Grid 2 with YAML

YAML is a way that we can take commonly used data and encapsulate it into a file, so that the details we might want to update, or run with several instances, can be sourced from one place. Here's the suggested example for our YAML file (called grid_configuration.yml):

There are two different configuration options that are available. If you have Selenium 1 Gris servers (i.e. Selenium RC) and you want to run them along with the more up to date Selenium2/WebDriver servers, you can. Define the details you want to configure in the YAML file, and when you start up the Grid hub, give it the flag -grid1yml and the name of the YAML file, and you're good to go. 

A  quick look at the config options shows us our grid1yml file configuration options (note that we haven't created a grid 2 configuration file yet):

The key takeaway here is that a lot of the parameters that we would want to keep track of can be housed in a couple of YAML files, and that makes configuration changes quite a bit easier to track.

Actually Running Tests in the Grid

Up to this point, I have to say that my update speed has been very slow with this chapter. That's been a combination of work conspiring against me, plus some unusual issues with getting files to compile that made me decided that I had to get to the bottom of it. Through that process, i discovered that Java 7 requires a little more verbosity than Java 6 did. I could be wrong about this, but on the bright side, I think I may have found an answer to the issues of not being able to compile in previous sections.

First, let's talk running tests that leverage the grid. This is a simple example, simply because I decided I didn't want to fight multiple machines, multiple drivers, and multiple error messages just yet (hey, I'm having such fun with it, why shouldn't I let you all do the same ;) ?). But, to show that we are actually "gridding" it, here's a simple test that I put together to make sure that it goes to a particular location. In the example below, I have three servers. One's running on Windows, one's running on Linux and one is running on Mac. I want to make sure the test in question goes to the Mac instance. 

Right, so what's interesting about the above code? Glad you asked. As I was setting this example up, I kept getting the compile error "SetPlatform requires Platform, but found String".

This is so similar to all of the design pattern errors I was getting, I decided that there had to be a reason I'm seeing this. Additionally, I've gotten to the point that I couldn't believe that so many code examples were just not workable. I have met David, and I know he's way more conscientious about this stuff than to allow so many "mistakes". Well, what if it's not a mistake? What if Java 7 requires stricter type usage? As soon as I thought about that, and remembered back in my old C days that I could "force" a type casting to be applied, I started experimenting. In the process I came up with the following approach and change (note the difference between the two files):

Once we forced the type requirement, the project compiled! Not only that, but it actually ran the test to the specific server we wanted to have run it. Look below, the grid console shows the test instance being run. It's the muted firefox icon, and yes, that server is indeed the one running on my Mac:

Oh, and note... I finally got my Windows Server to declare itself an Internet Explorer only node, so yes, it's doable. I don't know what I did differently other than have the Windows version run the latest version of Selenium Server... hmmm, that might be it.


Yeah, this section took a long time to get through, but on the plus side, I made a neat discovery, and that discovery may go a long way to solving a few other problems. Net result is that I'm going to go back and do some re-testing and re-coding of the previous examples and bring them in line with Java 7 realities. Seriously, I am ecstatic that there is a potential solution to all of the random compile errors, and if it's this simple, then there will be some easy changes to suggest to Packt so that they can perhaps make an update in the errata to reference Java 7 type conversion (or lack thereof :) ).

End of Section.

Thursday, February 21, 2013

Wait... You're Not Supposed to be Here!

Since I've changed jobs, I am no longer commuting to San Francisco, but instead I am commuting to Palo Alto. Changing from North to South, from getting on the train in San Bruno to getting on in Millbrae, and of course, no longer getting off at San Francisco 4th and King, but getting off at the Palo Alto Caltrain station.

Having done the commute up to 4th and King for a few years, and making the same general walk, you start to notice things, and then those things blend into the background and you just don't really think about them any longer... until something shows up in a totally unexpected place. In this case, that something totally unexpected is a lady that I often used to see on the walk from 4th and King up to 4th and Folsom. She had a distinctive tan patchwork leather blazer that she would wear very frequently. Over a few years I'd see it and think "oh, there's the lady with the patchwork leather blazer"... and then, over time, I wouldn't think about it any longer, because it was just there. She walked the same commute path I did, and so, meh, nothing out of the ordinary.

Yesterday, as I got off of Caltrain, I saw her again. The same patchwork leather blazer. In Palo Alto. Huh?! It caught me by surprise, and at first I thought "naah, must be a coincidence". Today, as I got off Caltrain, there she was again. Same blazer, and as I would see, walking roughly the same commute direction I was going once again. This time, I had to ask (and run the risk of being considered some kind of creeper, but oh well, whatever):

"Excuse me, but did you used to work in San Francisco?"
"Actually, I still do, but sometimes I work down here in Palo Alto, too."

We had a quick conversation, and she told me where she worked and how the company has a number of offices, and she sometimes goes to San Francisco, and sometimes goes to Palo Alto. Not a big deal, really. After a laugh and some quick small-talk, we went our separate ways.

What makes this story, I think, somewhat interesting is that I was prompted to start a conversation because I noticed something "out of the ordinary". This is a person that I had seen for years on my daily commute, and never said a word to. Why would I, really? It's kind of odd for a married guy to approach some random lady on the street to start up a conversation (or maybe it's not, and that's just my traditionalism showing through). It took seeing her in a totally unfamiliar situation to cause my curiosity to get the better of me and have to ask. It all came about because, at that moment, my first thought was "hey wait... you're not supposed to be here!"

How often in our testing do we miss interesting pieces of information because, in our world view, they've just always been there. We've gotten used to seeing them there, and so they must just be part of the ordered cosmos, doing its thing. It often takes us seeing something familiar in an unfamiliar setting to make us "wake up" and do some inquiry. That's normal, and I'm not suggesting that we suddenly start automatically questioning everything we see, or everyone (for one thing, it's a good way to get people to consider you a class A nutter, and I already have to fight that depiction enough as it is). Still, when you see something out of the ordinary, it might be fun to check up and see what you discover. In this case, someone I've seen for years, and had zero interaction with of any kind, now knows who I am, and I know who they are.

That's kinda' cool :).

Wednesday, February 20, 2013

The Benefit of "Intern Fresh" Eyes

It's interesting when a story or set of stories comes together as a big mass, and just by their very nature, they are manually intensive. Just by the virtue of their design, they defy automation, they defy fast checking, they require the physical patience and calmness needed to work through them, one at a time, for a long as it takes. I'm living that dream right now (for some definition of dream ;) ).

Seriously, we have a lot of stories that just fit a similar theme, and to help get through them, I've been assigned my very own intern for the next several days. Nice! He's a good guy, and I'm happy to get some extra help for a bit. Today, we set things up and we worked as a pair for much of the morning. As I explained the rules of the road for the testing we need to do (this is accessibility stuff, so it depends a lot on hearing speech from screen readers and talking into microphones for dictation), it was interesting to see how he interacted with the product, what steps he took, what frustrated him, etc.

What I love about "fresh eyes" is the fact that, often, they do not have any paradigm to hang things off of. In testing, they don't know the rules of the road at all, and that's great. It lets me see how someone would approach a problem without even realizing the problems that are already there. As I am going through and trying to determine what aspects of testing to teach for SummerQAmp, these opportunities are great, because I can often see very different approaches being taken, some very much out of the realm of what I would do because I'm both experienced and, let's face it, a little jaded.

As we were working through some of the challenges, I found it interesting to see why he would go in certain directions, what interested him. What he would do to try to make things work if at first they didn't (which, with a tool like Dragon, is really easy to do).

A few years ago I wrote a post about Pairing with An Expert to help open your eyes to those areas you might not be aware of. Just as powerful is the opportunity to pair with a domain neophyte, since their "ignorance" is a valuable tool... they don't know the rules, so they can much more easily break them, and teach us something new along the way.

PRACTICUM: Selenium 2 Testing Tools Beginner's Guide: Mobile Devices

This is the next installment (in progress) of my PRACTICUM series. This particular grouping is going through David Burns' "Selenium 2 Testing Tools Beginner's Guide".

Note: PRACTICUM is a continuing series in what I refer to as a somewhat "Live Blog" format. Sections may vary in time to complete. Some may go fast. Some may take much more time to get through. Updates will be daily, they may be more frequent. Feel free to click refresh to get the latest version at any given time. If you see "End of Section" at the bottom of the post, you will know that this entry is finished :).

Chapter 7: Mobile Devices

As David rightly points out, Mobile is the current "killer app" for our times. Some might even say it's the "killer paradigm" in the sense it is adding a whole new slew of approaches and considerations to testing. This section will look at how to get into and use three mobile platforms; Android, MobileOpera and iOS.

Note: The tests in question will be using emulators for this chapter. Real app testing, however, should be performed on real devices (the emulator will only tell part of the story).

Creating an Android Emulator

The following example requires three things. First, you need an Android emulator, and to do that, you can download the Android SDK from:

I called my dowloaded directory android-sdk to be consistent with the book

Next, open a shell window and cd to android-sdk/sdk/tools. Type the following:

./android create avd -n my_android -t 1

‰ "–n my_android" gives the emulator the name "my_android".

‰ –t 1 tells it which version of android to use. "1" gives us Android 4.2.

Note, the book says to enter "14", this will not work with the current distribution.

$ ./android create avd -n my_android -t 14
Error: Target id is not valid. Use 'android list targets' to get the target ids.

$ ./android list targets
Available Android targets:
id: 1 or "android-17"
Name: Android 4.2
Type: Platform
API level: 17
Revision: 1
Skins: HVGA, QVGA, WQVGA400, WQVGA432, WSVGA, WVGA800 (default), WVGA854, WXGA720, WXGA800, WXGA800-7in
ABIs : armeabi-v7a

Entering "1" will get the default version, which is Android 4.2.

STadmins-MacBook-Pro-2:tools michaellarsen$ ./android create avd -n my_android -t 1
Auto-selecting single ABI armeabi-v7a
Android 4.2 is a basic Android platform.

When prompted "Do you wish to create a custom hardware profile [no]", hit the return button (ie. accept the "no" option).

Do you wish to create a custom hardware profile [no]
Created AVD 'my_android' based on Android 4.2, ARM (armeabi-v7a) processor,
with the following hardware config:

Run the emulator with:

./emulator -avd my_android & 

If all goes well, you will see something that looks like this (note, it'll take some time to load, be patient):

Installing the Selenium WebDriver Android Server on the Emulator

Next, we need to load the android-server[ver].apk onto the Android emulator. Downloaded the APK file from

Start the emulator.

Go to android_sdk/sdk/platform-tools and run "./adb devices")

$ ./adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached
emulator-5554 device

Note: My serial number is "emulator-5554" your serial number will probably be different from mine.

Install the server:

$ ./adb -s emulator-5554 -e install -r /Users/michaellarsen/selenium/android-server-2.21.0.apk
2602 KB/s (1881490 bytes in 0.705s)
pkg: /data/local/tmp/android-server-2.21.0.apk

On the device you will see this (I put it on the main screen for effect, you'll see it listed in the apps section, though: 

Creating a Test for Android

Those examples we used in the last chapter should work with a minor change. In this case, that change will be to use AndroidDriver. There are some steps to do before that can happen:

Start the WebDriver server on the device. After experimenting with some errors, I restarted the emulator, and then ran the server process from the command line in the android-sdk/sdk/platform-tools directory as follows:

$ ./adb -s emulator-5554 shell am start -a android.intent.action.MAIN -n
Starting: Intent { act=android.intent.action.MAIN }

Next, forward HTTP traffic to the emulator.

adb -s emulator-5554 forward tcp:8080 tcp:8080 

With that, let's try a test we're already familiar with...

All we did was add an import statement and change the webdriver to point to AndroidDriver. Let's click Compile (success!) and let's click run test...

Success!!! Cool, we just ran a test on an Android. Well, Android Emulator, but good enough for a first step. 

Working With OperaMobile

As I've come to discover over the past few years, While Opera has a dedicated fan base, they are not really a big player overall in the browser and Mobile world. Still, even though they are a small niche player here stateside, they have put a lot of time and emphasis into their tools, and really, their tools are quite nice. What's more, they have made a really strong push and commitment to making their products very responsive to Selenium, so that alone makes them deserving of a look.

Setting up OperaMobile is actually pretty easy. They have an application package that can be installed on Darwin, so if you are using a Mac like I am, setup is a breeze. Just go to, get the OperaMobileEmulator.dmg file, double click, put the OperaMobile application into the Applications folder, and that's it. The default location for the application is /Applications/Opera Mobile Mobile Emulator .

Once you have set up OperaMobile, you can open it up and take a look at the rather large number of emulation modes it supports:

Here's an example page with Amazon Kindle Fire HD 8.9 selected as a destination device.

Making a test is much the same as previous examples, in that we are defining in the setup stage what driver will actually open. In this case, driver will open OperaMobile, with some parameters that David has provided, including a path to the executable (which, in this instance actually compiles :) ).

From there, we can compile (success), and run our test (not successful):

So this will take some playing with, I guess. The OperaMobile profile chooser launches, and if we select something, we get a browser. We have to do that select choice, though, and I suspect that may be part of the problem. Also, we get the OperaMobile browser to appear, but the get command is not posted, and the test times out. Again, something seems to be missing from these steps. The question is what?

Next steps will be to try to figure out what's happening with OperaDriver, but I may just forge ahead and start looking at iOS and the iPhone/iTouch/iPad framework.

Working with iOS
So I have to say at this point I'm a bit disappointed. No, it has noting to do with David or his explanations. It has more to do with Apple and their need to provision my emulator, and charging me $99.00 for the privilege. In short, that annoys me and frankly, I'm not willing to do that at this immediate point in time. I may well address this later on, but for my purposes right now, I'm willing to bypass this for the time being.

Still, I read through the details, and found out much of the information is very similar for what to do to set up and start driving Android and OperaMobile. Put simply, you fork Selenium from git, you build selenium to run in the simulator or on an iOS device, and then you set up your environment (for a fee) to be provisioned to run applications. From there, create a test like we have for the rest of the examples, with the following changes.  

1. Add an import statement that points to:

import org.openqa.selenium.iphone.IphoneDriver; 

2. In the @Before setUp() block, put the following line:

driver = new IPhoneDriver();

With all things being equal, that's pretty much all you have to do to make a test that will run on iPhone.

So there you have it. Much of the details are setting up the environments and geting the code needed to run inside of the environments so that they can be controlled by WebDriver. The cool thing to see is the fact that so little "code" needs to be written at the test level to make these changes. Creating and maintaining tests to be run on multiple different browsers and environments, it's really nice to see that there is very little that one has to do to make changes.

End of Section.

Tuesday, February 19, 2013

Weekend Testing Americas #37: Mobile Usability

Yep, it's that time again, time to get the word out about our next session and what we are going to cover.

One of the things we tried last time, and it seems to great effect, is to do some sessions that are broader reaching, focusing on bigger topics, and giving people a chance to plan for them and to participate. Usually, we keep the sessions kinda' hush hush, so there's no "favoritism" for certain participants, but we decided that, with some topics, there's really no need for any of that.

On Saturday, March 2, 2013, fledgling facilitator Justin Rohrman will be leading us in a discussion and testing session around Usability. The preliminary notes for this session, courtesy of Justin,  are below. If you find them helpful, please let Justin know. Also, if these early bird posts are helping to encourage you to participate in these events, please, let us know that as well :).

Weekend Testing Americas #37: Usability testing On Mobile Devices

Date: March 2, 2013

Time: 10:00 a.m. Pacific, 1:00 p.m. Eastern
Suggested Mission and Charters: Explore the Facebook Mobile interface and specifically test for usability

Assumption: Attendees must have Facebook running in a mobile environment. Native or mobile browser versions are fine.
Topic: LearnabilityPick an area of the app you are not familiar with (pair up however makes sense. Spend some time getting familiar. Discuss your learning experience. How did you learn to use the software? What made the experience difficult? What made the experience easy? What would make the software easier to learn?

Topic: MemorabilityFor your group, describe the part of the software you just learned. Was the functionality easy to recall? why? why not? What would make the software more memorable?

Topic: Error RateDescribe the errors you made while learning. What lead to you making these errors? What helped you to not make errors? What improvements would help reduce user error rate?

Topic: Efficiency Were you able to use the software in an efficient manner? describe what made the product efficient or inefficient for you. How would you improve the efficiency?

What did we learn in this session?
Would anyone describe usability testing in a different way after the hands on exercise
What kinds of tests can you apply when you go back to work
Was this a useful session


Monday, February 18, 2013

Up In the Air

One of the things I like about the scouting events that we do is the fact that we can get out and try some things that are a little out of the ordinary. Now that my son is a bit older, and a little bit "been there, done that" I often find ways to get my daughters in on the act, too. this past weekend, my youngest daughter wanted to come with us to the rocket launches that were happening at Moffett Field (these launches had been scrubbed twice, so this time was the charm). the plus side, this was a chance for my daughter to try out a few different rockets of different power levels, see how they performed and see how she would adapt launch to launch.

She came ready to go with three rockets. A small A class rocket (the Pink Dink), a medium grade B class rocket (the Riptide) and a more powerful C class rocket (the Chrome Dome).

We had some fun staging each of the rockets, getting them checked in, packing the cylinders with wadding and seeing how much was necessary, how much was too much, and how much would turn out to be too little (more on that later). As we got the rockets checked in, each rocket would be given a launch assignment (meaning a numbered pad that they could set up on; there were thirty pad running.

As we went through the day, we tried a variety of the rockets to see how they responded, and what would be the best setup for each. Early on we realized that the A class rocket wasted a lot of energy just getting up the support pole. A simple solution? Take a clothes pin and put it near the middle of the support rod. This gave the rocket enough stabilization to get a true flight and less fuel was wasted getting up in the air, which made for higher and better sustained flights.

The B Class rocket worked fine as is, and there wasn't much we had to do to get it up and into the air. three pieces of wadding, a folded up parachute, and an engine with an igniter was all that was needed. It flew flawlessly each time.

Our C class rocket proved to be interesting, in that it flew much higher than the other two. It also deployed way up in the air, but as we watched it, we noticed it was coming down really fast, as well as a long way away from the launch pad. we went to go retrieve it and after a bit of a walk (compared to the other rockets), we found it. It was sitting in the grass between the two main runways at Moffett field, standing at about a 15 degree angle. One of the fins stuck the landing (none broken, thank goodness) and the rocket was, all told, in perfect shape, save one thing... the parachute was nowhere to be found. Just the strings remained, and all the loops were in place. This could only mean one thing... the chute must have burned up during deployment and recovery. We thought three pieces of wadding would be sufficient. Apparently for a C class engine, a bit more is needed.

I'm glad that she was able to have this experience. She had a lot of fun, and it was really great to see her respond to the challenges, fix them, and launch the rockets again. I explained that, like in testing, we often try things out as experiments. Sometimes they work the first time, but often, they require adjustments, patience, and some last minute fix-it to make everything work. I have a sneaking suspicion this will not be the only time we do this :).  

Friday, February 15, 2013

The Dreaded "Third Week"

I've recently started working with two of my friend's kids. They are young, i.e. six and nine, and they want to learn music. Specifically, the six year old wants to learn how to play guitar, and the nine year old wants to learn how to sing.

As I've been working with them over the past few weeks, I knew that this most recent lesson would be one of the toughest, and I was 100% right. Not because I'm teaching anything hard, or because it's something difficult to understand... nope, it's because this would be week three, and if history has taught me anything, the "third week realization" would make its appearance... and sure enough, it did.

Here's how it works. In the first week, everyone is excited, it's new, they are looking forward to learning, and we're just getting to know what each other expects. The second week, we give them something to hang it on, something relevant that they can compare to. The third week, all comes crashing down, usually. Why? They realize now what is required of them, and the level of practice needed to do even the most basic of things. I introduced a drill to the guitar student and showed what needed to be done to "master" the drill, and as we worked through it, I saw it in his eyes. His fingers on his left hand hurt. His right hand was getting tired. He was slouching and grabbing the neck of the guitar with a death grip. He was frustrated. This part HURTS! It's not enjoyable. Most of all, it doesn't sound like music. It's trills and hammer-ons, using different fingers, and it sucks. The sad thing is, without learning this, I can't really teach him anything of interest.

My vocal pupil, likewise, after getting over the amusement of my asking her to repeatedly yawn-sigh, and sing through a few chromatic scales, found it frustrating that she could not sing along to Tears for Fears "Shout" (picked because it has a very simple melody line). I saw it in her eyes, too. She was struggling, she was wondering why she couldn't hit the notes. She looked at me with a share of disappointment and bewilderment. Again, I had to show her that these annoying little drills were necessary, that she had to learn where her voice actually was so that she could use it to vocalize correctly and accurately.

I could have been all business, and just shrugged it off and said "keep practicing, it will get better", but I remember many times doing the same thing for things I wanted to do in the past, and not getting any encouragement or a pep talk of any kind, and I subsequently abandoned a lot of those things. Maybe there's a better way to do this.

I decided to share with them my own experiences of the dreaded "third week", and to let them know that this was perfectly normal, but to also show them that they could get through it if they really wanted to. I stopped and asked them why they wanted to play guitar and sing. Was it just something they wanted to do to impress their parents, or was it something they wanted to do for themselves? If it was something they wanted to do for themselves, I asked them what would be a "quick win" for them. I'm a firm believer in quick wins. If a quick win for the guitar player was to put one finger on a single string and slide up and down to different note to make a melody, then cool, do that. For the singer, I asked her to find her favorite song, and I'd do whatever I could to help her sing it. I pulled out my bass, something I hadn't played for, at this point, several months, and goofed around with them a bit, got some laughs out of them, and then showed that we all had that dreaded "third week" in our pasts. In short, this is the hinge-pin moment for most, when they will decide to tough it out, or quit. Getting a "quick win" at this point is vital.

In testing, I think we often face similar situations. There's a lot we realize we don't know and need to know to be effective in a particular sphere. As I've been going through Selenium 2 Testing Tools, I've found a few times I've had to stop what I was doing because things weren't working the way they should. That's frustrating because, often, you don't know WHY they are not working. You're missing some of the details, or you don't realize that there's some library somewhere that you need to access or have an understanding of to use the item effectively. I've probably spent more time going over remedial Java in the past two weeks than I have cumulatively over the past eight years. More than a few times, I've thought to myself "aw heck, why am I subjecting myself to this?"

The reason, for me, is I've publicly put myself on the line to do it. I made a pledge, and in that pledge I said I'd allow myself to be either brilliant or stupid, enlightened or foolish, knowledgeable or ignorant. Even with that caveat, I still want quick wins to help me know that I'm on the right track, or that what I want to do is at least conceivable if not immediately attainable. It's no different with other endeavors. If we get something we can smile about, or makes us feel like we are making progress, we are much more excited about pushing forward. Take the time with the people you work with to get them to relax, drop some of the frustration, if possible, and share some of your own screw-ups and frustrations with them. Trust me, it makes that "third week" go by much smoother. Time will tell if my advice and approach will work for these two :).

Thursday, February 14, 2013

PRACTICUM: Selenium 2 Testing Tools Beginner's Guide: Working With WebDriver

This is the next installment (in progress) of my PRACTICUM series. This particular grouping is going through David Burns' "Selenium 2 Testing Tools Beginner's Guide".

Note: PRACTICUM is a continuing series in what I refer to as a somewhat "Live Blog" format. Sections may vary in time to complete. Some may go fast. Some may take much more time to get through. Updates will be daily, they may be more frequent. Feel free to click refresh to get the latest version at any given time. If you see "End of Section" at the bottom of the post, you will know that this entry is finished :).

Chapter 6: Working With WebDriver

The whole point of running Selenium is so that we can interact with and work with different browsers and tet to make sure that interactions behave as they should, items appear where they should, and that our applications work in the environments that we want them to. Selenium WebDriver is designed to work with the major browsers currently on the market, so this chapter will show us how to interact with Chrome, Firefox, Internet Explorer and Opera. Each browser requires an executable be installed on the system, as well as the browser in question (the Firefox driver is already bundled with the Java client; it's what we've been using thus far in the examples):

Using Firefox Driver

So far, this is the approach we have been using up to this point. The example below shows a standard JUnit test:

This example shows the bare basics of a test with no modifications.

Firefox Profile Preferences

In many cases, there are functions in the browser that a user may or may not want to have configured while they run their tests. To take advantage of this, a variety of profiles can be created, and the particular profile instance can be loaded and used. To do this, we:

  • instantiate a Firefox Profile object and then update the settings.
  • pass this object into FirefoxDriver where we instantiate it. 
  • load the profile with your details you have set. 

Imagine that you wanted to have the book site be the startup page for Firefox. Below shows us how we can do that:

Another example that David shows us is that we can locate the firefox binary explicitly. By entering the path to the binary, we could load the version of firefox we want to use. The syntax would look like this, alas, I can't get the system to compile if I use this option:

We can also point to a plug in using the following syntax:

Alas, this too fails for us, and for the same reason. In fact, many of the items in this chapter have the same "fatal flaw". I think they look like great ideas, but why oh why can't I compile them?!

Working with ChromeDriver

The whole point to WebDriver is to allow us to use multiple browsers, not just Firefox.

The next example shows us how we can load the ChromeDriver, and thus connect with Chrome. For this purpose, I took a copy of ChromeDriver and put it in /usr/bin so I could always rely on its position. With that, we compiled and ran, and lo and behold, here we are:

There's a fair amount of red down at the bottom, but don't be alarmed, that's the output from ChromeDriver. The thing to focus on is that ChromeDriver opens, and the test runs, and passes. Woot!!!

However, it looks like ChromeDriver and the extended examples suffers from the same issues as FirefoxDriver. Try as I might, this won't compile :( (this is the example to explicitly call out the file location. IT's complaining that it expects a string, not a file. Remove the File declaration, and it does the same thing. I'm at a loss. Java peoples, please tell me, it it me, or is something missing from this example?):

Working with OperaDriver

Opera, likewise, has a driver that lets WebDriver access and run it. Let's have a look!

To load Opera, we need to get the operadriver.jar file and for good measure, associated it with the project. Once we do that, it's all good to go. Granted, the test need some tweaking. Look for the spelling errors for the words "Chapter 4" (should be "Chapter4") and "index" (should be "Index"). Once you fix those, hey, you're golden:

Well, OK,; not completely golden. Those astute viewers will notice I did a little cheat here. I set the assert value to be zero. This means it will return a true even if the element isn't there. Why would I do such a thing? I wanted to show a completed test. Alas, the element I'm looking for stubbornly refuses to respond (it may be my system settings, I'll tweak with them some more later). Just to show you the full story, set the asset value to equal one, and here's what I get, at least as of right now:

Opera also has a profile that can be adjusted. The below example turns off geolocation, and then runs the same test as before:

As we can see the test passes (but I set the size to zero to get it to do that. Again, I need to poke around just a bit more).

Working with Internet Explorer

First off, I need to say how exciting it is to see this next step. One of my frustrations of two years ago was that Selenium RC was having major issues with running on 64 Bit Windows 7 at the time. Today, WebDriver does not have that issue, and the steps to get it up and running were actually fairly easy. For those who want to get up and running on Windows 7 64 bit, here's all it takes:

  • Download Intellij IDEA and install it
  • Install Java SDK
  • Download Selenium-Server-Standalone-x-xx.jar
  • Download JUnit.jar
  • Download IEDriverServer.exe
  • Link up the IDE to find the Java SDK folder, plus add selenium-server-standalone and JUnit jars as global libraries
  • Put the IEDriverServer in a place where the PATH variable can find it (or add it to your path).
Once you do that, create your Java class file, put in some simple commands, including calling the InternetExplorerDriver class, and let it run. If you do as above, this should be what you see (along with an IE browser running the commands):

OK, wow, that was pretty cool. That's four different browsers we can drive and little in the way of tweaking top get them to do their things. I must admit, I'm a bit frustrated about the File and Dictionary calls, as well as the compilation errors. I followed all of the steps as requested but alas, many of these examples were for naught. I really would like to understand why they are not doing what they are supposed to do, and if there's a way to make these work as printed. Again, is this because of me using Darwin and not Linux? Is it something special about Java 7? Do the examples rely on something that is now deprecated? Either way, we were able to set something running for all four browsers, and the fact I could get it to work on Windows has me somewhat elated.

End of Section.