QA Craftsmanship, Unite - Live From Pay Pal, it's yet another Meetup!!!

Live from San Jose, it's... me :).

This time, I'm excited to say that there's a new group that's formed in the Bay Area called "QA Craftsmanship". It's meant to be a corollary to the Software Craftsmanship movement, and it's looking to be a way to bridge the gaps between the programming and testing realms. That's the theory, in any event. Whatever it's called, I'm happy to see dedicated testers who are looking at ways to up their game, and the fact they are doing it here in the Bay Area, double bonus!

This one came quickly together, as I got the invitation on Saturday to join the group, so here's a quick run-down on what we are talking about tonight:

The Althea Studies Platform is a smartphone and web based SaaS platform for creating longitudinal healthcare studies, i.e. studies that collect data over a period of time. Study administrators can define studies on Althea's web site and deploy the studies as native applications on the iPhone and Android and also on the web for use by study participants.

Learn how automated testing gets down with AltheaHealth's mobile tracker (Android/iOS) using cucumber & calabash

Truth be told, I've had much more experience with desktop and traditional web apps than with mobile, and Calabash is totally new territory for me, so I'm curious to see where this talk goes.

Melvyn is starting out with a quick intro to Cucumber, and explaining how the gherkin interacts with the step definitions as defined in selenium, capybara, etc.

Melvyn likes to use some basic rules when it comes to writing tests. One example he points out is "If I can click it, it must be there", so put into a more direct focus, don't code specifically to items on the screen, code based on what you want to do. Defining things by their actual structure (buttons, links, etc) means that if those elements change or are removed, feature files will need to be modified. I like this approach, and think it could save a lot of editing in the future.

At this point, Mishal Shah picked up the talk and carried the discussion forward. Using Jenkins, we can use a variety of plug-ins, and in this presentation, the Android Emulator can be set up as one of those plug-ins. You can also plug in you actual phones into the Jenkins server if it's available, or via port forwarding, you can remotely connect your actual Android device into the Jenkins server (ok, that's a trip, I've gotta' try that out :) ).

Calabash-android basically lets you run Cucumber features on an Android device or emulator. It is controlled from your command line, and as you run your tests, you can see in real time the steps run on the Android emulator. The Feature files and test steps work just like we would expect Cucumber to do on our desktops. 

Again, just as with their web counterparts, remember to make a distinction between creating step definitions that drive behavior vs. driving interactions with elements. All of the options hat we are familiar with when it comes to Cucumber, i.e creating hook files, making macro commands, and other options that we either love or hate about Cucumber and it's underlying structure, Calabash does very much the same thing.

One interesting thing with calabash is that they are using Frank, which allows users to look and see the element IDs and such. Frank will let the user get every element id currently loaded. Key takeaway and urging at this point; don't look for text, look for element ID's (it will also save me from having to go and change tests if the text is changed later).


query, touch and a variety of other commands will allow the user to directly interact with the elements and be able to enter text, select items, etc. This talk is also looking to be developed out to be a hands on workshop in the next few weeks. Hoping I can be there, but if not, I plan to spend some more time in the calabash-android github repository.

One other interesting tidbit (this may be common knowledge to many others, but hey I'm new here) is that calabash has a very limited scope as to what the framework will interact with. It's limited to the apk file and what it will generate. You can't jump over to another app, it's out of scope. That's a security measure, so that you can't hijack other applications. I find that interesting.

Oh, and while I'm on the topic of "interesting", I took some time tonight to give out some "Ministry of Testing" stickers, and for those curious, though I doubt it's the first one in the USA, I think this may well be the first one out in the wild in the Bay area other than me ;):

Thanks for joining me tonight, I hope this was interesting (I certainly found it to be :) ).

Adventures in Context: What Can A Water Bill Tell Us?

I've posted a few of these in the past, and I have to give my family credit, they put up with a lot from me, but last night gave me a chance to explain to my kids a bit about the exploratory testing process, proposing ideas, and setting up experiments.

What could prompt such a discussion, and why would I bring such a thing up? This time, it was one of the first things that I heard walking into the house. after the "Hi, how was your day, we all did…" Christina looked at me with a less than thrilled expression, and said "we got hit with another HUGE water bill!"

At times like this, there's basically three things one can do. We can go into a fetal position (metaphorically), we can irrationally rant about the unfairness of it all… or if you live with a tester, you might go through something like this (which, roughly, is how last night played out)...

So, let's take a look at the bill… looking at previous months, yes, we definitely are considerably higher. This bill represents July and August, which was summer, with most of us at home or in the house for a much greater period than usual. That might be a good gauge. Let's think of the places we use water… showers, washing dishes, watering plants, bathroom breaks, getting ready in the morning/evening, cooking, plus three fish tanks, not to mention laundry, cleaning, washing cars... it can certainly add up, but does it add up to this?! In previous years, we've had spikes during the summer, but this looks unusually high.

It's usually about this point that I might normally make a variety or pronouncements and recommendations, but I know they will last about two days and then everything will slide back to where we were before. Hmmm, what if we did something different?

"Hey, who'd like to try an experiment with me?"

Truth be told, to some members of my family (especially my kids), this is the most dreaded statement any of them can hear. It has even more weight than "you're grounded". The reason? They know that, when I say "let's try an experiment" I tend to hold them accountable, and they actually have to do things and face hard facts with cold, uncaring data starting back at them. I intend to actually see if they can learn about their real behaviors, not just their imagined ideals of what they might be.

Me? I'm a huge fan of "X-diaries", with the X being whatever it is I want to measure. In this case, water consumption. What does a typical person like me do in the course of the day, and how much water do I consume? Only one way to find out for sure, and that's get into a mode of "write down every water interaction I can think of, and measure it, if I possibly can, or make a rough estimate otherwise.

Little experiments like this typically have me reaching for a notepad, pen and a stopwatch or timer. The reasoning for this is that I want to see if I can spot patterns, or interesting aspects I might take for granted or not notice in my everyday usage. One example is the shower. I turn on the water, and wait for it to be warm enough, and then get in. I don't take long showers because, frankly, I don't have the regular hair maintenance of normal people (of course, I spent more time in front of a mirror with a razor and shave cream, so it kinda evens out for time, but water wise, quite a bit less ;) ). Still, even at the best and fastest practical shower that I can take, the water still needs to warm up, and that takes about 45 seconds. 

Hmmm…

What if I were to take a bucket, and see how much water flowed out of the shower head in that 45 seconds? Better yet, what if I then used that water afterwards to water the plants? Over the course of a week, how much water would that represent? Could it allow me to turn the automatic sprinklers back one minute per zone? Could we use the water for something else, like for rinsing dishes after meals?

It's about this point in these conversations that I am either hit with MEGO (My Eyes Glaze Over) from my kids, or given a litany of reasons why it couldn't possibly work, or how it was a waste of time and effort, etc. Typically, at this point, I also tell them that I have very similar conversations with fellow testers and with programmers as well. In this case, though, I'm not talking about water distribution and use, but about software and how I test it. With this example, I showed them a little glimpse into how I often formulate tests.

First, I need a problem. Something that either is surprising, or comes across as out of the ordinary. A high water pill can be easily compared to web site performance or some other form of interaction that causes someone to say "wait, that can't be right".

Often, we only have hunches to go on, so we try our best to see if we can gather data, even if the data we gather is from a very small area. Reviewing HTTP headers and response times through Chrome tools is very similar to capturing the water in a bucket while it goes from cold to warm.

Once we have some data that we can look at, we can make some hypotheses and make a basis for further interactions or experiments. They may prove to be right, or they could prove to be wrong. If I perform elaborate and detailed captures of water and then transfer it to other uses, I may see total water usage go down. I may also not see any appreciable difference in water usage at all. That might mean that I am just transferring the use in equilibrium… or maybe there's a deeper problem.

Last year, as we were trying similar experiments, we discovered that a sprinkler valve had rotted over time, and that a slow leak had appeared How long the leak had been happening, we had no idea, but we fixed the valve, and for a time, that seemed to make a big difference in water usage. It's entirely possible that we may have a similar situation happening now (our house is just shy of 60 years old, so anything is possible ;) ). 

The point I wanted to show them, though, was that we can use our testing methodology and apply it to a host or problems in our everyday sphere, and likewise, similar challenges in our everyday lives can help inform our testing. My kids often laugh and say "Dad can turn anything into a testing problem". They weren't too amused last night, though I certainly was.  I'm not quite sure if these little talks of mine with my kids will encourage them to test, or will cause them to run with all power and haste to any career where they don't have to hear this crazy testing stuff. Time will tell I guess ;).

Book Review: HTML5 and JavaScript Web Apps

One of the factors facing anyone who wants to develop a web site or a mobile web application today is that the world of server-side scripting for functionality is (has?) giving way to much more dynamic applications with the client doing most of the heavy lifting. JavaScript has moved beyond being a little bit of glue code or a few dispersed libraries that added some special effects or eye candy to a site. Now, applications are being developed and generated by JavaScript libraries and frameworks to the point where static HTML may be slim to nonexistent.


The goal today is less a matter of control via the server and being open to creating applications that are performant, scalable, and utilize the latest Web infrastructure and application features. We are reaching a point where, for many, the mobile web is the way that they interact with the Internet (lugging around a laptop like me for much of the day is starting to look a bit old school ;) ).


"HTML5 and JavaScript Web Apps" by Wesley Hales is a survey of the emerging and solidifying standards that are defining the landscape of what web and mobile apps will look like in the near future. Yes, the emphasis here is mobile, though there are also several sections devoted to traditional web sites and applications. This books is, as I mentioned, a "survey"; it does not go in depth on any of the technologies. What it does provide is a lot of examples and ideas to spur thought, and an invitation to go and explore what's out there.


If you are a new web developer, or have not already spent significant time playing with front end development practices, this is not going to get you there. That's OK, though, it's not meant to.  If you are a software tester who would like to have a handy reference, and some context, as to what these features are and how they work, at 156 pages, it makes for a handy "cheat sheet" and jump off.


Several approaches to how the web developer can improve performance and user experience are mentioned, including tools that help with compression, web sockets, utilizing Web Storage, and the variety of JS frameworks that are currently in play (node, angular, backbone, ember, batman, and knockout get specific mention and coverage).

While the book's title talks about being about HTML5 and JavaScript Web apps, a more appropriate title would be "The Emerging HTML5e Standards and the Options it Provides, Coupled with Several JavaScript Framework Engines and Additional Utilities to Help Design for Both Mobile and Traditional Web"… but that would be really long ;).


So if this book for you? If you are looking for a quick survey, with some supporting information of the various "cool new bits" to play with, and you don't mind a lack of detail or depth, then yes, this book fits that role rather handily. Testers, if you want to get an overview of what a lot of these technologies actually do, and how to recognize them in the wild, then yes, this book is worthwhile, and even more so in eBook format, since the links jump you to other resources. If you are looking for an in-depth book that will give you practical hands on experience and exercises to work through to learn these technologies, then no, this book would be wholly inappropriate for that purpose.

Parallelization and JavaScript: SF Selenium Meetup, Oct. 2013

Hey all, TESTHEAD is live at the San Francisco Selenium Meetup, and looking forward to another night of pizza, ginger beer and talk about testing in Parallel and testing JavaScript. What are YOU doing tonight ;)?


The following are from the Meetup site, so consider this pre-amble. You'll definitely get plenty of my ramble in a bit.


Optimization through Parallelization (David Borin)

Dramatically improve the scale and speed of testing through the use of parallelization techniques. This includes writing test modules that can be run independently, doing as much preloading of test data as possible, and using parallel test runners.

Cross Browser Compatibility Testing JavaScript (Alan Parkinson)

People often run their WebDriver based functional test suites across multiple browser combinations with the aim to test cross-browser compatibility. Functional tests provide poor coverage of JavaScript and long test execution times. I will demonstrate how you can reuse your existing JavaScript unit tests with Karma and utilizing your Selenium WebDriver test infrastructure for providing fast cross-browser JavaScript compatibility tests. If you're not already doing JavaScript TDD then the additional benefits highlighted in this talk will encourage you!

My thanks to Lithium Technologies for providing the space an the food. Speaking of food, I'm going to go get some now. Be back in a little bit.

Alan Parkinson is first out the gate with a talk about how to parallelize tests for JavaScript. The app that Alan's company creates is, by his own admission, mostly JavaScript. A key truth that Alan wants to get out to everyone... You don't need to run your Selenium Test suite with every browser (*). What?! Heresy, you say! Well, whats the reason we are testing against all the browsers? Is it rendering compatibility? Is it compatibility with the DOM? Is it to get lots of javaScript code coverage? According to Alan, no, there was about two thirds of their code that wasn't even being exercised. So how do we do this? According to Alan, the answer is is to create unit tests for your JavaScript... and run *THOSE* tests cross browser. Hmmm.... OK, I can see the logic to that.

Alan makes an interesting point when he describes the options between functional tests and JavaSctipt unit tests. An average functional test suite takes about five minutes to run. The JS unit tests... take about 11 seconds. Alan did mention that he had a qualifier, and here it is... not everything is going to be picked up by unit tests. There's far too many JavaScript files, and there's actual behavior aspects that unit tests will not actually be able to pick up. So some tests will have to be run cross-browser, but the point is, not as much as we might think.

So what test runner do they use? Alan talked a bit about Karma and their "Spectatular Test Runner for JavaScript". There are some challenges, to be sure, so to solve some of their challenges, the made a Karma WebDriver Launcher, which allowed them to leverage their Selenium WebDriver infrastructure. Alan took  a time out to run through Karma and show how they have set up their framework and which options to call. Granted, it's a small set of tests for the demo, but no doubt about it, it is remarkably fast. Running the tests in parallel on multiple browsers becomes a quick option compared to native Selenium WebDriver functional tests.

Karma provides code coverage, but to do that, you need to instrument your JavaScript (in their example, they do this with Istanbul). They can also generate code coverage reporting formats and the ability to come in and see what is actually covered and where.

Overall, the main takeaway that alan wanted to deliver was that functional tests are an expensive way to get browser compatibility coverage. By comparison, JavaScript unit tests can provide much more coverage, much quicker, and the beauty of it is... the programmers do the lions' share of the work with this model, since they would be doing it anyway. Again, interesting approach, and a clever approach to cross-browser testing (and to be clear, they do still run a number of functional tests cross-browser, but the total number of tests is significantly less, since they prefer letting the unit test framework go for it with the code coverage and atomic level functionality. Again, interesting idea, definitely want to explore the idea more :).
  

OK, next talk is up, and now it's time to talk a bit more about parallelization... imagine being able to run ten thousand test modules in under five minutes. The simple answer is that parallelization is necessary to get those numbers. Spin up a battery of virtual machines, spread the tests equally over the machine battery, and get your CI server to do all the work (living in a world that uses Jenkins and AWS, I can confirm that this method is real and does work.

This approach does require that the effort be made to make tests as atomic as possible, and that dependencies are, if not completely eliminated, at least minimized as much as possible. Also, you don't need hundreds of machines. With 4 servers, you can create a Jenkins master, a slave with 10 executors, Selenium Grid hub and selenium grid nodes with 15 browsers. I'll leave it to you to do the multiplication on that. The point is, with that setup, you can do a fair amount of parallel testing.

Several great takeaways for the talk were encapsulated in the "mantras" slide, so rather than repeat them, hey, it's right here :):

Tools for parallelization?

Wow, that went quickly! Glad to have the opportunity to come out and play once again. Lots of interesting ideas, and I definitely want to take a closer look at some of the ideas from these presentations. If I had to pick any one thing to consider, it would be to see how many tests are "overloaded" or have a lot of steps. Could those tests be puled apart? Would it make sense to run them separately (and subsequently, more in parallel)?

And with that, I bid all a good night. Happy testing :)!!!

Learn a Scripting Language: 99 Ways Workshop #89

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.

My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions. 

Suggestion #89: Learn a scripting language. Use it to automate repetitive tasks or processes. Manipulation of text or data files for example.

I'm going to do something I don't normally do.. I'm going to punt. Well, not really. See, the thing is, I wanted to see if I could put together a workshop in a simple blog post that would discuss the nuts and bolts of scripting languages and why you should learn them. 

What I realized the several times I was trying to write this post was that this is, for real, not a topic that can be summed up in a simple 800-1000 word blog post. In fact, I could barely sum it up in a 50 post blog series on "Learn Ruby the Hard Way". Shell programming, scripting languages of any kind, any flavor... just take work, persistence and a lot of continuous and steady practice.

Thus, it's with that in mind that I am going to talk about what you can do, and share some sites and suggestions I have found to be helpful along the way (from someone who, admittedly, isn't much of a programmer, but I do what I can with what I do know ;) ).

Workshop #89: Download and install three different scripting languages, your choice. If you are using the bash shell or another variant, consider that one of the three. Find and address ten different challenges in your everyday work life that you can experiment with (data files, log files, web page downloads, etc.). The catch? Try to use each of the languages/approaches to solve the problem. 

This is what I refer to as the "Secret Ninja Cucumber Scrolls" method. Why? Because they were the first that I saw that made a point to take a topic (in this case Selenium and web automation using Cucumber and various tools) and made it so that each example was made to work in three different environments. There were advantages and disadvantages to each approach, but by the end of the process, each environment was able to accomplish the same task. What's more, if the user in question worked through each of the examples, they had a better appreciation of just how they could engineer a solution in not just one way, but multiple ways. 

What makes three the magic number? I think because there's enough natural variety out there with three options that it forces us to "think differently". When it's just one option, we do what we have to and if it works, great. With two options, we have to compare why it works better than another system. With three systems, we have to focus on what it is we are trying to solve, and see if we really understand what we are working towards, enough so that we can address it in what will likely be three unique ways. 

So with that in mind, realize the scripting languages I'm suggesting are not necessarily the "best" languages, but they are ones I would suggest for the fact that there's lots of avenues out there to learn them and frameworks to put into place and experiment with (that, and they happen to be ones I have personal experience with. If others you were expecting to see here aren't here, it's because I either don't know enough about them, or don't use them enough to be able to talk about them with anything resembling half a brain)...

JavaScript: At this stage, just about every site that teaches programming has a section on JavaScript, and frankly, it's the most ubiquitous language on the net, and will likely become even more so in the coming years. More than just JavaScript, there are frameworks that you can learn as well once you cover the basics. If jQuery, Angular, Ember, Backbone or Node are avenues of interest, then JavaScript needs to be a starting point.  

Ruby: Ruby is a stable language that has been around for two decades, has a rabid fan base, and is a platform of choice for many production web sites that use Ruby on Rails. It has a healthy ecosystem of tools and add-on "gems" that can be utilized to make many jobs a lot easier. 

Python: Another language that has been around for quite awhile ( I remember first hearing about it in the mid 90s), and has likewise become a popular choice for many testers due to support for Selenium and other testing frameworks. It also has a popular web site framework with Django, and the ability to build web apps and sites quickly and dynamically. 

bash: Wait, what?! You didn't think I was going to mention a handful of scripting languages and not include my own personal favorite, do you? Yes, I know that bash is technically not a "scripting" language, but work with me here. I think there is a lot of benefit to learning and practicing with the bash shell (or csh, ksh, zsh or whatever flavor you personally want to play with). Not only do you get to work with programming structures that are similar to the ones other scripting languages use, you may find that you can answer in one line of command line code what it might take dozens of lines to write in another scripting language (again, this comes back to the idea of "forget the tool, start with the problem"). 

Javascript (and various associated frameworks): Codecademy, Code School for the basics, with NetTuts+ being a good source for going into greater depth or viewing screencasts on more advanced topics and additional frameworks.

Ruby (and Rails):  Code School has a Ruby and Rails path that, while not free, is pretty comprehensive. Codecademy also has a track that covers Ruby well (just Ruby), and I would definitely mention Zed Shaw's "Learn Ruby the Hard Way" (you can look on TESTHEAD in the Practicum section and see my own adventure with this book and approach. I found it helpful, your mileage may vary :) ). Also, as Chris McMahon wisely pointed out in the comments, Brian Marick has a great book called "Everyday Scripting with Ruby" with a strong focus on using Ruby for software testing.

Python (and Django): Codecademy has a language specific Python track  and NetTuts+ has a compilation of "The Best Way to Learn Python". In it are several avenues that you can explore as you practice using the language. Also, though I cannot speak for it directly as I did the Ruby version, "Learn Python the Hard Way" from Zed Shaw is the blueprint for the Ruby book. I thought the Ruby book was very helpful, so therefore I'm extrapolating and guessing that, for those who want to use it to learn Python, it will also prove valuable to those willing to follow its approach and methodology.

bash: there's lots of places to look for this information, and lots of tutorials that you can dive into (as well as dozens of books), but I personally like the Linux Shell Scripting Tutorial and Advanced bash Scripting Guide. Alternately, a very nice book that gives a lot of great examples and possible jump off points is the bash cookbook (which I reviewed here).

Bottom Line:

Scripting languages, regardless of what you want to refer to them as, are programming. Let me re-emphasize that… 

You WILL need to LEARN to have a TOLERANCE for SOME amount of PROGRAMMING!

There's no way around it. This does not mean that you have to devote your life to programming, that you have to de-emphasize testing to program, or anything of the sort. You may, even after going through the variety of exercises for several months, decide that you just aren't cut out to be a programmer. If by "programmer", you mean a "production level writer of software programs that ship commercially", then you may very well be right. I'm certainly not. Fortunately, "programmer" means a lot more than that, and encompasses more than just shipping commercial software. 

Scripting languages (and there are many more that I didn't name here because of lack of my own interaction) have some advantages in that they can be executed and debugged more quickly than compiled languages can be, but the fundamental rules are still the same. Regardless of the language or methodology, some things are definitely going to need to be considered. You will make mistakes. There will be bugs. You will need to find them and fix them. You will need to practice. A lot. For a long time. That is, if your goal is to get good at using these languages. Sorry, no short cuts or simple solutions to offer here, just a lot of training for an extended period. On the bright side, you may find that you start to enjoy the journey ;).

Learn How to Use the Command Line: 99 Ways Workshop #88

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.

My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions. 

Suggestion #88: Learn how to use the command line. Shell and batch scripts too.

I have to admit, these are getting a little more difficult to write in this stretch. Suggestions 87 through 90 could keep people busy for years in and of themselves, and coming up with a workshop idea to put in a blog post is proving to be quite daunting. Still, it's worth a shot, and I'm going to give it my best.

Workshop #88: Spend some quality time inside the terminal program of your operating system. Learn its ins and outs, practice typing in commands and parameters that can help control and direct them. Gather commands that can be executed in sequence together (i.e. batch them). For extra credit, do it for both UNIX/Linux and Windows.

Really, this is not a simple "do this in an afternoon" endeavor. Windows may have a pretty graphical gloss to wrap itself in (Mac, too) but at their heart are command lines that can be executed. The command line provides a lot of power, flexibility, and some syntactical requirements that have to be learned. 

Command lines (and the various tools that interact with them), primarily handle four functions. Those four things are:

- variable interpretation and substitution
- quoting
- syntax interpretation
- redirection

That's about it, but "that's about it" covers a really large field. Still, by looking at it from that perspective, it makes a lot of things easier to understand, and thus, it becomes easier to see what we can do with those options.

When we see something like:

"cat file1 file2 file 3 | grep '[0-9]\{1,2\}\/[0-9]\{1,2\}\/[0-9]\{2,4\}' | uniq | sort > file4"

we are, effectively, calling on all four of the functions (though it may not be so obvious). 

In the above example we are saying the following:

- take the three files I have named, and concatenate their output (show the contents of the three files one after another) together as one output stream. 

- "pipe" that output stream (take the output of one program and direct it as the input to another program) to "grep", and have "grep" look for a string. If grep finds the string, it will keep it. if it doesn't, that line will be removed.

[Yes, that quoted option after grep is a regular expression, or regex, like we talked about in the last suggestion. They are hugely helpful in examples like this. In this case, all that it means is "look for any date string that is formatted as "mm/dd/yy" or "mm/dd/yyyy"]

- take the output that was parsed by grep, and pipe it to a program called "uniq". 

-  uniq will receive each of the lines, examine them, keep only the lines that are "unique" and remove the lines that are not. 

- uniq will then pipe its output to the input of another program, called "sort" which, as its name indicates, will reorder the lines so that they are in alphabetical order.

- finally, sort will take its output and redirect (write) it into a file (file 4), where the input stream will be stored and saved.

So there you go, redirection, quoting, and syntax. Variable substitution is going on in this example, too, albeit under the covers. How? Well, the file names we provide are variables in their own right, as far as the commands are concerned. 

If we wanted to make it a bit more apparent, we could save the same command as a script file, and provide variables like this:

file1="$1"
file2="$2"
file3="$3"
cat ${file1} ${file2} ${file3} | grep '[0-9]\{1,2\}\/[0-9]\{1,2\}\/[0-9]\{2,4\}' | uniq | sort > file4


If we were to save this simple script in a file like "strip-out-dates", make it executable, and run with the names of three files, the script above would execute and work on them, like this:

% strip-out-dates example1 example2 example3

When it finishes, you'll see a file called "file4", and that file will have a combination of all the lines from the three files that you fed to the script that had valid dates in them.

This, again, is just a little bit of what you can do, but suffice it to say that, if you wanted to take a variety of commands like I have here and run them, or stack a bunch of commands you use over and over, then you can build up scripts that can allow you to run commands in the order you have placed them in the file.

Bottom Line:

The command line can feel really daunting, but in truth, it only does what it is directed to do and what it has the ability to do. The shell itself is rather limited. The commands that it can run, and the permutations as to how those commands can be formatted, now that can go on forever. The good news is that, even there, there are also some basic rules that we can use that can help us. Those rules and examples, though, will have to wait for the next post.

Product Review: Acronis True Image 2014

One of the fun things about writing a regular blog is that, over time, you get some interesting opportunities come your way. Many of them, for various reasons, I've not been able to follow up on, but every once in awhile, something either looks interesting, meets a need, or a part of me that might otherwise be reluctant says "oh what the heck, why not?"

It was in this guide that I received a request from Acronis Software and asked if I'd be interested in checking out and reviewing their newest product, Acronis True Image 2014. I figured "sure, why not?" If nothing else, it may be an interesting experience to try something new, and see what I could discover in the process.

I decided that this might be a fun experiment for a few reasons. First being the fact that I've been playing fast and loose with my PC. I used to be much more meticulous about things like anti-virus software, firewalls, backups, imaging, etc, but it's been awhile (since the MacBookPro became my main system). Still, there's a lot of stuff I do on the PC that I'd be rather upset if I were to completely lose ,so I considered this a timely and fortuitous call to action.

Installation was relatively quick, even on my now somewhat long in the tooth Windows 7 PC (a 2009 era Toshiba Satellite for those interested). It was with this well seasoned machine that I figured a more nuanced and interesting test could be conducted.

There's two levels to this application. The first is the stripped down, almost bare bones display of the most critical functions. For those who want to get in, do what they must and then get out, this is a nice setup. Clean lines, uncluttered, easy to navigate and understand. Ah, but what fun is a backup and restoration app if it doesn't give fine control to the one setting it up? If you are one easily smitten by such things... Adornis has you covered there, too.

Some nice features that True Image offers are as follows:

 

Complete Control of the level and Density of the backup.

If you want to create a very specific and cherry picked set of files that you need to back up, along with a set of partitions on your disk, you can do that. The control options are geared towards the simplest and quickest path, and if you want to "set it and forget it", it's very easy to do so. If you would like to go in and perform fine control and very specific file level interactions, you can do that as well.


Individual file selection from back-ups.

In addition to backing up either full disks or individual file level, the user has the options of performing a file level recovery from a disk partition, if that is desired. rather than have to go and pull down a full backup, the user can go in and select a single file and just pull that one option. Yes, this has been available for systems for years, but being able to do it this simply and this directly is new and, frankly, kind of nice.


Create bootable media that can interact with the True Image backups.

This option still requires a bit of futzing with the BIOS to set the boot order, but if you do, then you are just a USB stick or a CD/DVD away from getting your system up and restoring a True Image backup. again, the system allows for both "drop in simple" and "fine motor control" options, depending on how involved or detailed you choose to be.


Back up to the cloud.

Acronis gives the user a number of "DropBox" level options for online storage, and in this capacity, users of Acronis can utilize the True Image equivalent of Dropbox. While drop box create and drag options are cool and widespread, True Image has added the ability of pushing and pulling full backup images from the cloud. this way, if you'd like to get a dedicated hard drive out of the equation entirely, you can certainly do that. In addition, users can also use the sync functionality with multiple devices and keep all of their items up to date. Click "Publish" and the whole world can see it, or just those who you designate access to.

Some other interesting tidbits:

Try and Decide - Curious to see if an app will cause problems? try and decide lets you give it a run, and if it turns out to be more trouble than it's worth, you can roll it back.

Secure Zone - creates a special secure partition for your backups on your disk.

Boot Sequence Manager - Lets you boot your computer from a disk image if desired.

Backup Conversion - Move your backups from windows to Acronis and vice versa

File Shredder and Other Security Tools - exactly what it sounds like


Bottom Line:


The tool isn't magic, but as far as having a balance between ease of use and flexibility/control, Acronis hits more notes than it misses. Systems are of course going to be limited based on the hardware are features. My test runs were not what I would call "zippy", but I am perfectly aware of the fact that that may have more to do with my own system limitations and hardware than anything having to do with the Acronis application.

Also, I've only spent a few days with this app, and there's so much that I'm curious to play with and tweak with. I suppose the best endorsement I can give is that, after several days of poking and prodding, I feel like I want to know more and do more. In these days of short attention span theater, that says a lot.

tl;dr: nice app, simple to operate for the basics, but plenty of firepower to geek out with if users choose to. A nice balance overall.

Learn How to Use regex: 99 Ways Workshop #87

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.


My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions.


Suggestion #87: Learn how to use regex.

All right, we're starting to get into direct recommendations that were workshop ideas from before. That means we get to start getting more specific.

I mentioned in an earlier post that shell scripts using regex can help make for dynamic tools to parse information and clean up output for later use, but I didn't say a word about what regex actually is.

For those that know all this already, this may be of limited value. If this is new, this is by no means an exhaustive treatment of regex, but I'd be remiss to not at least offer some nuts & bolts and examples, so that you can see why this might be something you'd want to learn more about.

Therefore, without further ado...

Workshop #87: Determine what tool(s) you would like to use to practice working with "REGular EXpressions" and use that tool(s) to practice using them. 

I've heard this said a number of different ways, and since we are reading it, it probably doesn't matter much, but this is a personal thing, and you can decide for yourself how or what you want to call it. I'm a firm believer in the idea that, if something is an abbreviation of a set of words, then the pronunciation of the words as a whole needs to inform what the abbreviation will sound like. Since we are talking about REGular EXpressions, you will always hear me say it as "REG-ex" (hard "g"), not REJ-ex (soft "g"). What you choose to call it is totally up to you ;).

Anyway… regex works by letting the user provide a rule, and that rule is meant to help identify a sequence of alphanumeric characters. If we choose to look for an exact word ("apple"), we can just use the exact word, and tools like grep, sed, awk, etc. will find the literal match of that word.

That's OK, but more often than not, we want to look for items that will be dynamically assigned through variables or have a variety of ways to be found, not just absolute terms or phrases. What do we do then, if we don't know in advance what the value will be?

regex to the rescue :).

Here's some very quick regex examples, written from the perspective of bash/linux tools:

- a period (.) matches any single character.  '….' would match any four characters

- 'A.' matches an “A” followed by any character

- '.A.' matches any character, then an “A”, then any character

- an asterisk '*' means 'repeat zero or more of the previous character'.

- 'A*' means zero or more “A” characters

- '.*' means zero or more of any character (letter, number, symbol, a blank line, etc.)

- '..*' means any single character, followed by one or more of any character (so an empty line isn't an
option here).

- '^' means the beginning of a line.

- '$' means the end of a line.

- '^$' would mean any blank line.

- '\' is used as an escape character. This means if you want to look for any of the above examples literally (., *, ^, $), you would use it first, like '\. \* \^ \$'

- '[abc]' is a "range". In this case, it means "find any 'a', b', or c', anywhere on this line. Using the [] range option, we can also use the '^', but in this case, '[^abc]' means "show me lines that do NOT have the characters 'a', 'b', or 'c'. All inclusive ranges can also use a shorthand like [A-Z] or [0-9].

- '\{n,m\}' is used in bash and Linux (and elsewhere) as a repetition option. If we see something like '[0-9]\{2,3\}', this would mean "show me lines for any numeric sequence that has two or three digits. If we were to see something like '[0-9]\{3\}', that would mean "show me lines with exactly 3 numerals in sequence".

OK, with that, let's try something a little more interesting.

$ grep '[0-9]\{3\}-\{0,1\}[0-9]\{2\}-\{0,1\}[0-9]\{4\}' datafile

What do you think this might be? As printed, it might be hard to tell, but if we were to look at each element separately, we can probably figure it out.

[0-9] any sequence of numerals between 0 and 9

\{3\} limited to three numerals exactly

- a literal dash character

\{0,1\} repeated zero or one times

[0-9] any sequence of numerals between 0 and 9

\{2\} limited to two numerals exactly

- a literal dash character

\{0,1\} repeated zero or one times

[0-9] any sequence of numerals between 0 and 9

\{4\} limited to four numerals exactly

If you guessed that this is a regex to help find a US Social Security Number, you are correct (this example comes courtesy of the "bash Cookbook" by O'Reilly).

There are many more options to regex. This just scrapes the surface, but even with just this level of understanding, you can do a lot. A full blown tuorial on regex goes well beyond the scope of these posts, but if you would like to have a general, all purpose tutorial on regex, check out http://www.regular-expressions.info/tutorial.html

Also, there are a variety of regex "engines" available. Linux uses the POSIX engine. Programming languages use a variety of standards, many of them similar but with their own individual quirks. If you are using programming languages like Python, Ruby, PHP, Perl, etc. you will need to look at how regex is implemented in your language of choice.

Bottom Line:

regex is a core idea in a variety of scripting languages, programming languages and methods to make shell scripts much more dynamic. They take time to understand, and like anything other skill, repetition and practice goes a long way towards a better understanding of how to use them.

Book Review: The Practice of Network Security Monitoring

This certainly fell into my lap at an opportune time. With the various revelations being made about the NSA and its tactics, as well as the upsurge in attention being paid to network and application security in general, this book was a welcome arrival in and of itself. 

There's a lot of attention paid to the "aftermath" of security breaches. We see a lot of books that talk about what to do after you've been hacked, or tools that can help determine if your application can be penetrated, along with tools and recommendations for performing that kind of testing. 

Less often asked (or covered) is "what can we do to see if people are actually trying to get into our network or applications in the first place?" While it's important to know how we got hacked, I'd like to see where we might get hacked, and sound an early warning to stop those hackers in their tracks.

To that end, Network Security Monitoring (NSM) makes a lot of sense, and an important line of defense. If the networks can be better monitored/protected, our servers are less likely to be hacked. We cannot prevent all breaches, but if we understand them and can react to them, we can make it harder for hackers to get to anything interesting or valuable. 

It's with this in mind that Richard Bejtlich has written "The Practice of Network Security Monitoring", and much of the advice in this book focuses on monitoring and protecting the network, rather than protecting end servers. The centerpiece of this book (at least from a user application standpoint) is the open source Security Onion (SO) NSM suite from Doug Burks. The descriptions and the examples provided (as well as numerous sample scripts in the back of the book) help the user get a good feel for the operations they could perform (and control) to collect network data, as well as how to analyze the collected data. 

The tools can be run from a single server, but to get the maximum benefit, a more expansive network topology would be helpful. I can appreciate that my ops people didn't quite want to see me "experiment" on a broader network for this book review. After reading it, though, they may be willing to give me the benefit of the doubt going forward ;).

There are lots of individual tools (graphical and command line) that can be used to help collect and analyze network traffic details. Since there are a variety of tools that can be used, the author casts a broad net. Each section and tool gets its own setup, and an explanation as to how to use them. The examples are straightforward and easy enough to follow to get a feel as to how they can be used.

The last part of the book puts these tools into action, and demonstrates examples as to how and where they can be used. The enterprise security cycle is emphasized (planning, resistance, detection, and response), with an emphasis on the  last two items. NSM uses its own process flow (collection, analysis, escalation, and resolution). By examining a variety of server side and client side compromises, and how those compromises can be detected and ultimately frustrated, we get a sense of the value and power of this model.

Bottom Line:

My approach to learning about NSM in general comes from being a software tester, and therefore I'm very interested in tools that I can learn and implement quickly. More important, though is the ability to apply a broad array of options. Since I don't really know what I may be called on to test, this varied model of NSM interests me greatly. From an understanding level, i.e. an ease of following along and seeing how it could work and where, I give the book high marks. I'm looking forward to when I can set up a broader and more varied network so I can try out some of the more expansive options. 

On the whole, "The Practice of Network Security Monitoring" gets the reader excited about getting deeper into the approach, and looking to where they can get more engaged. As tech books go, it's a pretty fun ride :).

How Does Your Testing Add Value?: 99 Ways Workshop #86

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.

My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions. 

Suggestion #86: See the bigger picture. How does your testing add value to your team, project, organization?

Wow, now that's a suggestion. It's also not a simple, cut and dried thing that we can talk about in a "do these three things and you will add value".

I can't give you a simple "go forth and prosper" kind of answer, but I can share some thoughts and realizations that have helped me decide what to do with this idea over the years.

Workshop #86: Run a regular inventory of your organizations values. What do they do? Who are their customers? Who pays the bills? How do you justify your cost to the team? Then work towards aligning your actions to the answers your get.

Whoa, dude! Isn't that a little out there? Maybe, maybe not. It's a set of questions everyone, in any role, should be asking. 

Contrary to popular belief, corporations and company's do not exist to hire people and pay them. We do not exist to be hired by corporations and companies to get paid. Yes, both happen, but it's a relationship based on a simple reality… companies exist to make money. 

If you are creating something that can be directly sold, and a profit is earned, then you are a producer, and you are what is referred to as a "revenue center". You directly help the organization make money. Unless you work for, or own, a company that directly sells software testing services, you are not a revenue earner for a company as a software tester. You are a cost. You may be a very necessary, very important, and very valuable cost, but make no mistake, YOU ARE A COST! Just accept that.

This is why it is critically important for software testers to really and clearly understand what they represent to a company or to a team. We are a hedge. We are insurance. We are information and analysis. Each of those things are vital, but they do not directly, unambiguously add to the company's bottom line. On the other hand, there is decades worth of evidence that shows our being there can protect revenue, and that company's with bad quality products can certainly lose revenue. Testers being there and doing good work can enhance product reputation (when we do our job and do it well). It helps us garner influence because our product is better than other options. 

A helpful, and sobering, comment was said to me about twenty years ago. It was when I was first at Cisco Systems; one of their senior engineers put it very succinctly: 

"We are not where we are today because we are better than everyone else. We are where we are today because we suck less than everyone else!" 

The point he was making with that comment was to illustrate that, yes, we had bugs, some of them quite bad, but we were doing a better job testing and finding ugly issues than anyone else out there at the time.

If we want to make clear how and where we can add value, we have to consider a number of important areas. For me, this can be summed up as follows:

- spend time with your customer support team (or at least their support system application) and get to understand the most sensitive and delicate parts of your organization. 

- read actual support requests and tickets, so that you can see where customers feel genuine pain

- make a test charter around those areas, and dig to see if what they are seeing is just the tip of the iceberg.

- develop domain knowledge of the industry you work in, and of the customers of your product. When I worked with Tracker Corp, that meant "learn as much about Immigration Law statutes as you can".

- make a point to be consistent, and provide clear and concise bug reports. Take the time to understand them and recognize which issues really rise to the level of "Oh, we have GOT to fix THIS!!!"

- pay attention to the conversations that executives have. Read through the press releases, shareholder meeting updates, anything that points to what the company wants to do both now and in the future. Get to really and fundamentally understand what they perceive as the roadmap.

- work directly with your team to cross-pollinate and share knowledge. Learn from each other and help others learn new tools, techniques and approaches.

- be willing to work yourself out of a job. Show integrity, purpose, and dedication. If there are better ways to do things, champion them, even if they might be seen as "but wait, this might lessen my own influence." It may, but chances are, you will have other opportunities to do other things.

- be honest with others, and most importantly with yourself. Know your limitations, as well as your strengths. Don't be coy about them. If you are struggling, ask for help. If you see someone struggling, help them.

Bottom Line:

Every company culture is different, every manager you ever have, and every direct report that ever reports to you, will be different and unique. At the root, though, they all want the same things, which is to do well enough to not have to worry, to have others treat them like human beings, to afford them the respect and dignity that goes with it, and to do work that actually matters. Ultimately, recognizing that, and working accordingly, will do wonders for your career outlook and engagement. In short, if you want to be valuable to your company, BE VALUABLE TO YOUR COMPANY! Know who you are, where you stand, and what you can do. 

From there, act on it.