Tuesday, January 29, 2013

PRACTICUM: Selenium 2 Testing Tools Beginner's Guide: Getting Started with Selenium IDE

Note: this is a somewhat "Live Blog" entry, meaning I'm going to update as I go through and have comments. These may be daily, they may be more frequent. Feel free to click refresh to get the latest version at any given time. If you see "End of Section" at the bottom, you know that that one is finished :).

Way back in the Fall and Winter of 2010, I introduced an idea on my blog called PRACTICUM. The idea was that I wanted to see if the exercises in a book lined up with a real world user, and what would I think of the outcome and examples? More to the point, would a book that looked reasonable on the surface actually work in real life for a user, perhaps one with not the exact same setup?

These were the ideas that went into the Practicum series I did, and the first book I tried to tackle was David Burns' "Selenium 1.0 Testing Tools Beginner's Guide". Things went swimmingly for several chapters, and then things started to go very wrong, to the point where I couldn't continue. For the record, it wasn't David's fault; the Windows implementation of Selenium RC w/ Java was flaky at the time, and I just plain got stuck. I've felt bad that I haven't gone back to rectify this for two years now, so I decided to do something about it, and that something starts today.

In the spirit of "a little code every day", I've decided to revisit this topic, effectively "mothball" the first books posts, and do the whole thing over again with David's Second Edition book, "Selenium 2 Testing Tools Beginner's Guide". Why? Lots of reasons.

First, it will give me a chance to see if running on a Mac w/ Darwin will work better than running on a PC with Windows 7.

Second, a lot has changed in the world of Selenium since 2010, and I want to see how much and in what ways.

Third, I've been meaning to do a proper review of Selenium 2 Testing Tools for months now, and having reviewed Unmesh Gundecha's Selenium Testing Tools Cookbook made me decide this was as good a time as any.

Finally, the PRACTICUM posts force me to be honest. I can't just skim a book and say "hey, this was good" or "well, it didn't quite jive with me". Nope, I have to bite the bullet, work through each example, in somewhat real time, and talk about what's good, what sucks, where I'm brilliant, and where I'm stupid. Probably multiple times of each in the same post :).

Thus, without further ado, PRACTICUM commences once again. Thank you, David Burns, for being the inspiration.

"Selenium 2 Testing Tools Beginner's Guide"

For those who have the first volume, I will say that David has faithfully stuck to the same format as the First Edition, and this helps a lot for those who have read the previous volume; it grounds everything in the same context, and it makes me go "oh yeah, I remember that!". The examples also roll a lot faster this time, too, for the same reason. thank heaven for familiarity :).

The Preface is a good place to start, as it tells us what we are going to need to be successful. Again, I'm slightly off the mark because I'm not using Ubuntu, I'm using Darwin, but it's closer to the mark than I was the first time with Windows 7. For those wondering what you will need, and where to get them, start here:
ETA Aug. 6, 2022, since Firebug is no longer maintained (and many of these links are out of date), I'm also adding this link):


Installing everything is quite simple. Run the steps that each tool tells you to, restart your system or browser if necessary, get this done first thing so you're ready to go when you need to.

Chapter 1: Getting Started with Selenium IDE

OK, yes, I know some of you are already rolling your eyes… that whole "friends don't let friends record and play back" bit, I know... well ,*can* it! At least for a little bit. I said I'd review the book serially and that's exactly what I'm going to do. David chooses to start out the book with the IDE in the first few chapters, so IDE it is.

The first step, Install the IDE

[Simple enough, really, just go to http://seleniumhq.org/download/ and pull down the most recent version (which as of this writing is 1.10.0). Likewise, you could just open up Firefox, go to add-ons and do a search for Selenium IDE. Select it, install it, restart the browser, and there you are.]
Next step, Record our First Test 

[ Hit record and go to http://book.theautomatedtester.co.uk/chapter1 
Click on the radio button.
Select "Selenium RC" from ther drop down box.
Click on the "Home Page"link. 
 Stop recording. 
Click Play on the IDE.

What you get looks like this:



Again, not setting the world on fire, but simple first steps help get people to be confident and try bigger things. The point to this is that something is happening, and we can actually see it doing its thing.]

Updating a test to assert items are on the page

In the previous examples we were able to interact with certain elements. This may be enough for some tests, but what if we want to make sure something is there that we are not specifically calling out in the test steps, like a key word or value?
We can do that with assert or verify. An assert will test to see if a value exist. If not the test ails and stops. If verify fails, a failure is marked but the test will continue.

Selenium IDE adds a context menu to Firefox. right-click (or Control click on Darwin) to see it.

[ Example:


  • Start recording.
  • Navigate to http://book.theautomatedtester.co.uk/chapter1.
  • Select "Selenium Grid" from drop-down.
  • Verify "Assert that this text is on the page" is present by selecting "Verify TextPresent Assert that this text is on the page" (ctrl-click on Darwin). 
  • Hover over the button that says "Verify this button he here" is on the page (seriously, it says that ;) ). (ctrl-click and select "verifyElementPresent with the target verifybutton".
  • Stop recording.
  • Click playback.
See the following:



Again pretty basic, but getting a little more interesting.]

Adding Selenium IDE comments


If you want to insert a comment, it's easy to do so:



  • In the current test, ctrl-click the verify step.
  • Click on Insert New Comment.
  • Click on the Command textbox and enter in a comment.
  • Run the test.

And there you have it. ]

Multiplying Windows

What do we do when windows proliferate (pop up windows, different screens, etc.)? We keep track of them by using child processes. Selenium IDE helps us do that in the following manner:
  • Open up Selenium IDE and go to the Chapter 1 page on the site. 
  • Click on the second line that has the text "Click this link to launch another window". 
  • When the window appears, ctrl-click over the top text and select "VerifyText id=popup text". 
  • Click the "Close the Window" text. It will ,well, close the window. 
  • Stop Recording.
  • Press Play.
It looks like this:


This is deceptively simple, but it's very cool. What we are seeing is the fact that, by tagging the individual pop up windows and giving them unique ID's, we can manage the flow of the test, direct our processes to the correct windows, and do so in the right order.

Here's how to do it with two pop-up windows:


  • Click record. Go to the Chapter 1 page on the site.
  • Click on "Click this link to launch another window" link (the first one). This will launch a pop up window.
  • (ctrl-click), and select "assertText".
  • Go back to the parent window, click on the second "Click this link to launch another window" link. This launches a second pop-up window.
  • Verify the text on the page.
  • Move to the first pop-up window. Click "Close the Window" and close it using the close link. Notice the "clickAndWait" command.
  • Move to the second pop-up window. Click "Close the Window".
  • Go back to the parent window and verify an element on that page.



Working with AJAX 

The next example shows us an initial AJAX test.


  • Start up Selenium IDE, start recording. 
  • Go to http://book.theautomatedtester.co.uk/chapter1. 
  • Click on the text that says "Click this link to load a page with AJAX". 
  • Verify the text that appears on your screen. 

What does running this test tell us?


Hmmm, that doesn't look right. Good news, though, at this stage, it's not supposed to. We need to include a command before the verify text called "waitForElementPresent". See below (also, I changed the assertText to verifyText because, well, I picked the wrong thing the first time. Hey it happens :) ):



A number of these commands are run implicitly when other commands are being run. An example of this is the clickAndWait command. This will fire off a click command and then fire off a waitForPageToLoad. Another example is the open command which only completes when the page has fully loaded.

This next example takes a look at what we need to do to use another AJAX control and wait for the element to appear. Same steps as before:

Fire up Selenium IDE and set it to record.
Click on the load text to the page button.
Navigate to http://book.theautomatedtester.co.uk/chapter1.
Click on the "load text to the page" button.
Wait for the text "I have been added with a timeout"

Note, steps 1 and two are backwards in the book. Not a big deal, pretty easy to figure out what to do in this case. What's not so clear is that the output does not look the same. This is what happens on the page when we follow the directions:


This is what the book says we will see at this point.


and this is what we actually see:


Putting in the waitforText command isn't a big deal, we just need to wait for the text to load, highlight it, and then select the waitForText option from "Show All Available Commands". Except that doing that actually gives us the following:


It works, but for those looking for the values in the original example, they won't show up with the directions as described. You have to enter those values manually, like this:


Hmmm... when running it as depicted, the example fails. Firebug tells us html5div is indeed the value for the div, but it's not responding. This example, however, does respond:


Calling out "id=html5div" explicitly solves the problem. This may just be a difference between IDE 1.5.0 and 1.10.0. I'll check later :).

The point with all of this, really, is to talk about the waitFor* commands, and what they can do. Look through the value list to see all of them.

Storing Elements From the Page

this example demonstrates how you can create variables to store text that appears on a page without actually having to specifically print the string in the test. This one is also a good example because it constructs the test without actually recording it, but letting you access elements and place them into the IDE sequentially.

To see how this works lets work through the follow example:

  • Open up Selenium IDE, but don't record this time.
  • Start with the base URL of http://book.theautomatedtester.co.uk/
  • Click on the first row, In the Command box, type "open". In Target, type "/chapter1".
  • Go to the chapter1 page in the Firefox browser, and highlight the text "Assert this text is on the page".
  • CTRL-click and select the "Show All Available Commands" menu, and select the storeText option (which has already associated "id=divontheleft" as the target for this action).
  • A dialog will appear. In the text box, enter "textOnThePage". This will then be stored as the value for the storeText command.

  • Click on the empty row below the storeText command in Selenium IDE.
  • Type "type" into the Command box. 
  • Type "storeinput" into the Target box. 
  • Type "${textOnThePage}" into the Value box. 
  • Run the test.
So what just happened? We took a value from the page, stored it as a variable, and then called on that variable to put the text into a different location (n this case, the text box on the page called "storeinput", and we placed the actual text that we stored in ${textOnThePage} into that textbox. You can see it below.

Debugging Tests

There are a few neat little things that you can do to walk through and see how thing will work in your tests. The first is that, if you highlight a row in the IDE, and press the 'x' key on your keyboard, you can execute that single step. You can also click the pause button after pressing play, and that will halt the test at that point, clicking the pause/replay button acts as a toggle, so that you can step through each case (adjust the playback option to get better control of this). Also, clicking on the Find button next to the Target text box will momentarily flash a yellow highlight on the page, showing you where that target is. If the IDE cannot find a target, it will print an error in the log. You can also use echo as a Command and put ${variableName} as the Target to see what the value of the variable is, like this:


Breakpoints can also be enabled at a certain line in the script. to do that, ctrl-click and select "Toggle Breakpoint". This will create a stopping point for your test.


Setting Up Test Suites

Selenium IDE allows the users the ability to go through and set up multiple tests and saving them as a group. To do this, go ahead and create a few different tests:

  • From Firefox Menu, select "New Test Case".
  • Create a few actions in the table (record them or enter them in directly).
  • Click "Save Test Case As..." and give it a meaningful name. Click "Save".
When you have a few tests saved, they'll look something like this:


Now all you have to do is press the Run all button and it will run through all of them in sequence.


What You Can't Record (at least not yet)

At this point, there are some things that record and playback just doesn't work with, as well as interactions that are just not supported with Selenium IDE. David gives us a brief list here:

  • Silverlight and Flex/Flash applications cannot be recorded with Selenium IDE. 
  • HTML 5 is not yet fully supported with Selenium IDE. 
  • Selenium IDE does not work with Canvas elements (you can't make tests move items around on a page). ‹
  • Selenium IDE cannot upload files. 

Some of these examples will have ways to deal with them later when we get into WebDriver.

Summary 

22% of the books total volume just got covered in this initial chapter! Granted, a lot of that was due to pictures of the IDE and what to see in the screenshots. Still, this is a pretty impressive overview; for the most part, everything as explained works right out of the box. There are some differences in the presentation format and some values, and how the tool responds to them, but again, I think a fair amount of that falls into version differences between what was available when the book was written and what's available now. Also, there's a couple of places where there are small omissions, and these I think fall into the category of "implicit knowledge" (highlighting text to actually be able to access the commands associated with that element. Easy to see once you've done it once or twice.

So far, so good!

End of Section

4 comments:

Anonymous said...

Luv the fact that you show what the book says you should see and what you really see. This is very helpful to that has Selenium in their path of fun.

Alessandra Moreira (@testchick) said...

Hi Michael, this is such a great idea! Coincidently I'm also reading Selenium 2.0, after having read Selenium 1.0 years ago. It will be great to follow your commentary as I go through the book! Thanks for sharing!

Tarzan said...

I purched the (Kindle) book and have had no luck whatsoever with the 'Multiplying Windows' procedure. I thought this was because I was running IDE on a Mac, but now I'm on my Linux machine and seeing the same thing.

It basically stops recording after I close the first pop up window, and then IDE is 'broken' from that point on and won't record anything new until I restart the (selenium) application. I click 'Record' and it turns (brighter) red, but no actions on FireFox are recorded.

Have you seen this behaviour? I must be doing something wrong, but I can't imagine what. I'd love to get some mileage out of this book like it seems you've done, but right now it looks like I'll have to stick with on line tutorials (that don't use this book ; ).

Michael Larsen said...

Tarzan, I've had some weird spots w/ the IDE as well. If you're not on the latest version, I recommend you update it and see if that resolves the issues. Also, most of the book ignores the IDE completely after the first two chapters, so don't get too discouraged. The IDE is a teaser to get you to try out the ideas. The real meat and potatoes happens with WebDriver itself and accessing it with your personal flavor of unit testing framework (for all the jokes about how much stuff you have to generate for JUnit, and some occasional errors you will see in the later chapters, the development solution David has proposed is pretty nice. Use tutorials for the IDE stuff if you want/need to, but it's really not emphasized at all later in the book.