Cross Browser Fun: Adventures in TestCompleteLand

As I said previously, I'm looking to see how TestComplete would work were I to try to use it to handle some of my regular or desired testing. That means handling my environment as is, which is that Sidereel is a Rails app. Some of the bigger challenges I currently face are that it's a bit of a headache to do a lot of cross browser testing with my current setup of Gherkin and Cucumber with Ruby (not impossible, it just requires some tweaking and making sure everything plays nice).


The benefit (ideally) of having a tool like TestComplete is that it is meant to be a self contained solution. While Selenium/Webdriver/Capybara/Cucumber/fillInTheBlank requires a fair bit of integration to get everything to play together, I was curious to see what I could do out of the box to focus on cross browser testing (and not just some random demo site, but my site that I have to actually test!).

Here's a basic example:

- Using TestComplete, record a test where I open a browser window.

- Go to www.sidereel.com

- Click Log In to get to the fields for the username and the password.

- Enter the username and password.

- Click Log In.

- Make a checkpoint to that we can tell that the page has the right username.

- Click Log Out.


The test runs, albeit the record process creates a bit of detritus that I then ended up having to strip out (wait states, hover points, mouse movements, etc.). Still at the end of the process, we have a working script.


So what does it take to make a cross browser test? This is pretty slick, to tell the truth. What you do is you create a 2nd test file, and that acts as a wrapper and a controller. You put the calls for the browsers you want to run, and then you call the test you want to have run/repeated. In this case, the login test can be set up to run on IE, Firefox and Chrome, and if we want to set up conditionals or loops, we can do that to run the tests as many times as we want to.

Showing a hiccup when we try to test all three browsers; the setup of each browser is shown at the bottom. 

Yeah, this is a simple and quick example, and it doesn't necessarily tread on a lot of new ground. I would be interested to see how well this setup tests and identified CSS variations, since that's something that I deal with regularly (especially between browser versions; would TC catch any differences, or would I have to eyeball that to make the identification and make the call of it being a bug or not? Something to think about :).

Getting My Bearings: Adventures in TestComplete Land

I had anticipated that there would be a few things I'd need to address to get to this in an effective way. I didn't think it was going to take me three weeks to get back here. Still, with a lot of the details of Test Coach Camp and CAST now completed, as well as taking a week out to go to Summer Camp (note, this is actual Scout Camp, not the SummerQAmp that you will likely be hearing a lot more about in the coming days), I think it's time to get back into the swing of things and visit with an old friend I really have not seen in a long time.

As I stated previously, coming back to TestComplete after what has been a few revisions (version 7 was the last one that I had actual interaction with, and most of my knowledge came from versions 5 and 6), I figured it would be like coming back to a familiar neighborhood a couple of decades after I'd been a regular. In some places, that's not so difficult (the Sunset District of San Francisco looks remarkably similar today as it did in 1990). Some neighborhoods, of course, will look very different (my home town of Danville, CA has changed greatly since 1990, so much so that there are whole neighborhoods there now that didn't even exist when I moved away). Where does TestComplete fall in this category? Surprisingly, a lot of the look and feel is very much the same.

This is primarily a Win32 application for Win32 applications. In the last few years, it has moved into the web space, so much of the infrastructure for this application is primarily of a Win32 flavor. While I would be really happy if I were to learn that TestComplete now supported Perl, Python or Ruby, alas, that is not to be, even in version 9. It does support VBScript, JScript, DelphiScript, C++Script and C#Script. These are languages that, admittedly, I'm not all that comfortable with, but they will work in a pinch and I can at least navigate them and make sense of what I see.

A quick example of a test script.

The system has a lot of flexibility and ability to integrate with and coordinate with Visual Studio, so if that is the primary environment that a tester wants to do their test development in, TestComplete is well suited to take advantage of this space.

What is also helpful is that TestComplete has enhanced and improved the Keyword testing apparatus and makes it a choice for recording and constructing tests. The benefit of Keyword tests is the fact that the tester can, with a little practice, construct tests using the keyword interface that will allow them to develop robust and effective examples that use such options as looping, branching, Data Driven Testing (DDT) and allow for modification of elements to work as standalone tests or as a suite of tests. If the user is so inclined, once they have cobbled together an effective set of tests, they can export the tests to one of the supported scripting languages.

A quick example of a Keyword test.

At first blush, the obvious approach will be to use the recording method. For people just getting started, this is a natural place to start, and several tests can be created in this manner. However, just as I would tell anyone using the Selenium IDE or any other tool, friends don't let friends use Record and Playback. Once you have built a number of keyword tests and have become familiar with the method for creating the building blocks for a keyword script, you may find that it is just as effective to build your scripts from scratch using the keyword tools, rather than recording and going in to modify the overly exact steps that are recorded.

A cool new tool that is new to version 9 is the Test Visualizer, that helps to show you, based on where in the script you are, what is happening. Rather than rely on the script and its description, the ability to see images that define exactly where you are in the script are available to the user. This is helpful when you are trying to look at recorded steps and debug/determine what you want to do as you plug in new script lines or keyword blocks.

The Test Visualizer with some sample SideReel screens

The biggest challenge with tools like TestComplete is that they start off gentle, they give some examples of what a tester can do, and then things get steep really quickly. The keyword interface helps, but even there, if you are a neophyte to the world of programming, you will struggle beyond the basics with TestComplete. The good news is that, even with that steepness, there is a lot of user documentation and screen casts to help the user make the move beyond the basics.

So as I stated at the beginning, this is a "get your bearings" level post. Over the coming days, I'm going to commit to a little bit each day rather than try to slam out big jumps. Also, I have some actual questions I want to answer, and I'm going to use my own site (sidereel.com) to try to answer them, or I might make some examples to go deeper if the options available don't carry over to SideReel. Stick with me over the next several days and see where this tour takes us. In many ways, your guess is a good as mine :).

A Tourist Returns: Adventures in TestComplete Land

I'm willing to bet some of you are scratching your head at the title.

Back in 2007, when I was working with Tracker Corp, we were exploring the options available for performing automated testing of both Windows applications and Web sites. We had received a number of recommendations, and I was tasked with trying out a few of the options. Ideally, we wanted to have "one tool to rule them all", and while we had some money we could spend on the process, it wasn't unlimited. Tracker's total headcount at the time was 20 people; we weren't looking to break the bank on this.

After several screencast demos, a variety of vendor conversations, and me playing with several demo releases and installations, I decided that I liked AutomatedQA's tool TestComplete. It had some cool options for providing what I liked to refer to as "computer aided testing" within a Windows app, and it let me "Record and Playback" a variety of basic tests.

Now, I need to make a few things clear. I was never an expert at TestComplete. It used languages that were fundamentally different than what our team was using, though VBScript was a close enough environment that I could use it to communicate with our team, who were using VB.net or C# for most of the application code we were writing at the time. Also, we had implemented our web solution using a tool called DotNetNuke, which had a stubborn tendency to resist robust automation efforts.

Still, even with all of that, I found quite a bit to like in TestComplete. Alas, my journey with TestComplate ended when I changed jobs. The TestComplete license stayed with my former employer, and I embarked on a new expedition into Ruby/Rspec/Cucumber Land, a realm in which I am dwelling and actively exploring today.

Ruby/Rspec/Cucumber and its assorted tools has been an interesting learning curve, and I'm doing quite a bit of testing with them, helped considerably by the fact that my development team uses all of the same tools. Yet I've long wondered "what ever became of my friends in TestComplete land?" How are they faring? What does TestComplete look like now?

The cool thing about having a blog, being a tester, and shooting one's mouth off at opportune times is that, amazingly enough, people will listen! It's with this that I was presented with an opportunity. Come back to TestComplete Land, and see what has changed in my absence. Oh, and would I be willing to write a bit of a travel guide and some comparisons to my current workaday environment, and some thoughts about where TestComplete has come since I last visited their fair land? I said "Of course!"... and this is where we are today!

So what's the catch? There isn't one, other than the fact that the folks at SmartBear have given me a license for TestComplete for free. In exchange, I write about it and tell them what I think. I'm mentioning this out of full disclosure to say that I have received the software so that I can write about it. I'm happy to do so, but I want to make clear I am not obligated to provide a glowing review. I'll call them as I see them, and if I get confused, lost, or frustrated, y'all will definitely know. Likewise, if I see something really interesting, noteworthy or cool, I'll likewise let you know.

This will be a series, and I'll be coordinating with Alex Forbes over at SmartBear over the areas to cover. Stay tuned for further travelogues :).

My First "Guest" Blog Post (For SmartBear) :)

For some people, this may not be a big deal, but I think it's pretty cool :).

I wrote a thing for SmartBear, and they posted it. It's just a simple user story about how I used TestComplete to drive an API tool.

There is a nice main point I made. Sometimes automation scares us because of the enormity of the task. When that's the case, step back and go small. Really small, in the case I used. Being able to string together many small "cuts" (small autonomous scripts) will often do more than trying to use a big weapon to slay a beast (i.e. a large framework of tests).

Anyway, my entry can be found here.

Standard Disclaimer: I do actually use TestComplete, and I think it's a pretty cool commercial tool. I received no payment or perks from posting my blog entry there, just so y'all know :).