Our Python "Esperanto Project"

Much of the time, my work environment is not pretty. It's not the elegant situations that are spelled out in books or in "best practices" guides. Often, there are things that would look convoluted to outsiders, that seem like strange and quirky paths to get to places and to accomplish tasks that seem, well, not at all ideal. Why do we use them? Because it works. More to the point it has worked for years, and the thought of ripping out everything and starting anew would be a tremendous loss.

I have been discussing with my daughter ways that we could get more involved in and create an environment that we can both use, both agree on, and both work in and understand what each other is doing. In other words, we both decided we would put together our own "Esperanto Project" to help each other learn interesting tools, try out various frameworks and have an excuse to apply the ideas we are soaking up here and there and put them into an environment we can both work with.

Currently, our project resides on a Trusty Tahr build of Ubuntu Linux. The agreed to language for what we are going to do is Python, mainly because my daughter and I are both roughly skilled at an equal level (somewhere between novices and advanced beginners). As a web framework, we are using Django, because, well, Python. Selenium WebDriver is installed, with the idea that test scripts will be written in, you guessed it, Python. For my own fun, I am adding JMeter, Kali Linux and a few other tools to practice testing scenarios and particularly to exercise APIs, utilizing Python as the scripting engine. Finally, we are using PyDev as a plug-in to the Eclipse IDE because, hey, why not ;)?

One of the reasons I want to do this is that I want to be able to not just play with tools, but also have a way to keep the things I learn and find in an environment that can follow me from place to place. Each company uses their own set of tools and languages, and it's likely that I will not be using Python at different jobs. That's OK, since the goal is to not necessarily do a direct port of what I do from one company to the next, but instead, get to a point where I am able to develop and test an environment with a broad range of tools and become more familiar with all the possibilities, while also teaching my daughter how these tools are used. In turn, I'm hoping she will be able to teach me a thing or two later on down the road.

I joked with Kristoffer Nord yesterday via Twitter that his "Python for Testers" course would perhaps be an ideal jump start to this goal of ours. I'm looking at how I can make this into something interesting going forward, and I'd like to make regular updates to it and say where we are in the process. More to the point, I'd like to use it as a chance to ask for help here and there from the broader community, specifically the Pythonistas out there, because our goal is to use Python as the unifier of all the tools we pick, wherever we can.

It may work well, it may work terribly, but we won't know until we try :).

Book Review: Python for Kids

As I have been working with my daughter to learn programming, we have been focusing on learning Python.  We chose Python primarily because it was something I felt we could both come at from a similar frame of reference. I had worked with Python but in limited capacity, and Amber had not really worked with any programming languages. By working on Python, we could be somewhat closely matched with what we were learning.

NoStarch Press has a number of books that are ideally suited for younger programmers, and to that end we both decided to look at the books together. We will be doing three “for Kids" books reviews during February, provided we can get through all of them. All journeys have to start somewhere though, and this journey will start with Python for Kids. I should also mention that, for a limited time, you can get Python for Kids as part of the NoStarch and Humble Bundle "Humble Brainiac Book Bundle" offer, which comes with a lot of other additional books.

Python for Kids is written in a way that lets the user, whether they are kids or adults, readily get into the material and see how the examples work and try them out for themselves.

Chapter 1 starts out with installing and using Python 3, so right away Amber and I had to adjust to the new syntax (Codecademy’s Python modules are still based on Python 2).  Be aware that there are some subtle differences with Python 3, one of the main ones being the parentheses required when using the print(“text goes here”) function (yeah, like that ;) ).

Chapter 2 introduces the user to calculate values using simple equations via operators, using parentheses to ensure the desired order of operations, and creating and using variables to store values for use in those calculations.

Chapter 3 introduces strings, lists,  tuples, and maps, each of which allows the user to manipulate text in a variety of ways. Each of the approaches has rules as to how to move items and how they are represented, ranging from simple strings with are just quoted sentences, up to maps which have keys and values that can be called as needed.

Chapter 4 introduces the user to drawing methods, and in this case, that involves using a module called turtle (which in turn uses a tool called tinter). Using this tool, we can actually draw the turtles movements with lines moving left and right, moving forward and backward, as well as starting and stopping with up and down commands.

Chapter 5 focuses on asking questions and responding based on the response the system gets. For experienced programs, this i about if statements and their related commands (elif, else)  and how commands allow for combining conditions (using and/or) and converting values (using int, str, float and none).

Chapter 6 talks about “Going Loopy” which, as you might guess, deals with handling loop conditions and repetitive tasks. for and while loops are covered, and break conditions are shown to, well, break out of loops.

Chapter 7 talks about ways to recycle code and reuse it defining and using Functions and importing Modules. Variable scope is also covered so as to see where values are assigned, used and passed back and forth.

Chapter 8 focuses on the things in python, otherwise known as classes and objects.  this chapter covered the details of classes and how they an be used to create instances, and how objects can be created, call functions to work with objects, and assigning object variables to save values in those objects.

Chapter 9 covers the various built-in functions available in Python, those that we can import into Python programs and use directly (functions like abs, bool, print, float, int, len, etc) as well as howe to open and manipulate files.

 Chapter 10 covers a variety of helpful modules, such as copy, keyword, random, shuffle, and the sys module that allows the user to control the Python shell itself, reading input with stdin, writing output with stdout, telling time with the time module, and using pickle to save user information in files (yes, there’s a module called pickle ;) ).

Chapter 11 revisits turtle, and how we can use it to draw a variety of items. including basic geometric shapes, changing the pen color, filling shapes and reusing steps in functions to combine drawing shapes and using multiple colors in one function call.

Chapter 12 covered using tkinter to make “better graphics” and my oh my does this take me back. I used Tcl/Tk back in the 90s for a number of things, and the syntax to use tkinter is very familiar.  Simple shapes, event bindings so objects can move when keys are pressed, using identifying numbers to move shapes and change colors are all covered here.

Part II gives the user a chance top practice the programming chops they have developed in the first Part by making a ball bouncing game called, appropriately enough, Bounce!

Chapter 13 starts out using the tinter module, in which we can create a class for a ball and move it around the screen.  Coordinates can be set so that it will bounce off of the top and corners of the screen, and a randomized option so the ball has a dynamic movement.

Chapter 14 continues with creating classes for the paddle, and using coordinates to see if the ball hit the paddle. Event bindings tie the left and right arrow keys to control the  paddle,  and make it so,  if the player misses the ball, the game ends when the ball hits the bottom.

Part III continues the ideas so far shown with “Mr. Stick Man Races for the Exit”.

Chapter 15 shows how to make graphical images of, you guessed it, a stick man (using GIMP to make the actual images) and making the backgrounds of the images transparent so they won’t cover up other things.

Chapter 16 covers creating a Game class making a background image. By using functions called within_x and within_y, we can determine if items have collided with other items. We also discover Sprites and a sub-class, PlatformSprite, to draw the platforms on the screen.

Chapter 17 walks through the details for taking the pictures we created for Mr. Stick Man back in Chapter 15 and setting up options for the figure to move, turn and jump.

Chapter 18 finishes off the Stick Man game, and shows that putting together a working game is not trivial, but it’s also not a huge endeavor either.  We focus on using the images we made for Mr. Stick Man so he appears to be running, use  collision detection to tell if he hits the sides of the canvas or another object, and have Mr. Stick man drop if he runs off the edge of a platform. We also make a door that Mr. stick man can go through, ending the game.

The book also provides an Afterward that discusses some additional options for doing basic game development Alice, Scratch and Unity3D, PyGame, which is a dedicated library for making games, band a discussion of other programming languages and a super quick look at what they do , where to get them, and what it takes to make a   “Hello, World!” program. The Appendix highlights keywords in Python, and the glossary defines a number of words kids might not hear in their everyday interactions.

Bottom Line:

Python for Kids clocks in at a little over 300 pages, but you won’t feel like it as you work through it. There are numerous small projects and samples to examine, and all of the solutions to the puzzles and challenges are available at http://python-for-kids.com/. While this book is geared towards kids, there’s plenty in here to keep adults engaged and focused. Often, programming books are plagued with the “implied knowledge” trap that happens so often, where small projects give way to bigger ones, without a clear understanding of how the jump was made. Python for Kids does a very good job of avoiding that problem. Having kids as the primary audience, they take their time explaining key areas, and yes, for programmers with a little work under their belts, this is going to probably seem remedial, but as an introduction to Python, I think it’s first rate.

If you are looking to give your kids a kick start in learning how to program, if Python seems like an interesting language to start with, and if you’d like to have a couple of tangible projects to play with and augment as you see fit, then Python for Kids should be on your short list of programming books… provided having fun while learning is a priority.

Django 1.7 and… ME (yet another Live Blog)

So this might seem an odd spot, but this has become something of a mission on my part for 2014. I’ve decided that I want to try to become multi-lingual when it comes to web frameworks. We have all sots of interesting frameworks to play with to make web apps and web sites,and Django is the Python centric web framework, and therefore one I want to know more about and get more experience with. Seems a great reason to come into San Francisco and see what the Pythonistas are doing and how they are doing it with Django.

Yes, this is going to be live blogged, and as usual, it may be messy at first. Forgive the stream of consciousness, I promise I’ll clean it up later :).


A bit about our topic this evening (courtesy of Meetup):

Django 1.7 is one of the biggest releases in recent years for Django; several major new features, innumerable smaller improvements, and some big changes to parts of Django that have lain unchanged since before version 1.0. Come and learn about new app loading, system checks, customized select_related, custom lookups, and, of course, migrations. We'll cover both the advantages these new features bring as well as the issues you might have when upgrading from 1.6 or below.


A bit about our presenter this evening (also from Meetup):

Andrew Godwin is a Django core developer, the author of South and the new django.db.migrations framework, and currently works for Eventbrite as a Senior Software Engineer, working on system architecture. He's been using Django since 2007, and has worked on far too many Django websites at this point. In his spare time, he also enjoys flying planes, archery, and cheese.


Lightning Talks 

#1 Randall Degges - Django & Bcrypt

Randall kicked things off right away with a talk about how Django does password hashing and securing of passwords, with the estimated cost o what it takes to crack a password (hint, it's not that hard). If you want to be more security alert, Randall is recommending that we consider using BCrypt. It's been around awhile, and it allows for transparent password upgrading (users update their hash the first time they log in. No muss no fuss :). Sounds kinda cool, to tell the truth, I'm looking forward to playing with it for a bit.

#2 Venkata - Django Rest Framework w/ in-line resource expanding

The second talk discussed a bit on the Django REST framework. Some of the cool methods to handle drop down, pop open and other events were very quickly covered, and some quick details as to what each item can do. A quick discussion with fast flashes of code. I caught some of the details, but I'll be the first to admit, a lot of this flew right past me (gives me a better idea as to areas I need to get a little more familiar with). Granted, this is a lightning talk, so that should be expected, but hey, I pride myself on being able to keep up ;).

#3 Django Meetup Recap

The third lightning talk basically covered a recap of what the Django group has been covering and some quick recaps of what has been discussed in the previous meetups (Ansible, Office Entrance Theme Music, Integrating Django & NoSQL, etc.). Takeaway, if we want resources after the meetups are over, we have a place to go (and I thank you for that :) ).

---

Andrew Godwin's Talk

This seems like a great time to say that I'm relatively new to Django, so a lot of what's being discussed is kind of exciting because it makes me fgeel like I'll be able to get into what's being offered without having to worry about unlearning a lot of things to feel comfortable with the new details. Part of the new code is an update to South (which, as is mentioned above, is something Andrew is intimately involved in).

Details as to how apps are loaded and how to check for and warn programmers about what may happen with an upgrade. Having suffered through a few updates where things worked, then didn't and not having any clue as to why, this is very appealing.

Another new aspect is an adjustable and tunable prefetch option, so that instead of all or nothing, there's a spectrum of choices that can be looked up and help based on context.

A rather ominous slide has flashed across saying "Important Upgrade Notes", and a new detail is that all field classes need to have a deconstruct() option. It's now a required method for all fields. Additionally, initial_data is dead. It's important to have modules use data migration instead. In short, don't automatically assume that older modules that use initial_data will cleanly work. I will take Andrew's word on that ;).

So what's coming up in Django 1.8? Definitely improvements in interactions with PostGreSQL, as well as migrations for contributing apps. But that's getting a bit ahead of the race at the moment. Expect Django 1.7 to hit the scene around May 15th, give or take a few days. Again, I will take Andrew's word on that ;).

---

There's no question, I feel a little bit like a fish out of water, and frankly, that's great! This reminds me well that there is so much I need to learn, especially if my goal of becoming a technical tester is going to advance farther than just wishful thinking or following pre-written recipes. It's not enough to just "know a framework" or "know my framework".

As was aptly demonstrated to me a year and a half ago, I spent a lot of time in the Rails stack, and then I went to work with a company that didn't user Rails at all. Did that mean all that time and learning was wasted? Of course not. It did give me a way to look at how frameworks are constructed and how they interact. I'm thinking of it like learning Spanish when I was younger. Don't get me wrong, I'm not great shakes when it comes to Spanish, but I understand a fair amount, and can follow along in many conversations. What's really cool is that that gives me an add on benefit that I can follow a little bit in both French and Italian as well, since they are closely related. that's how I feel about learning a variety of web frameworks. The more of them I learn, the easier it will be to move between them, and to understand the challenge that they all face.

In any event, this was an interesting and whirlwind tour of some new stuff happening in Django, and I plan to come back and learn more, with an eye to understanding more next time than I did today. Frankly, that shouldn't be too hard to accomplish ;).


Thanks for hanging out with me. Have a good rest of the evening, wherever you are.

Sikuli: Some First Impressions

Today marks a transition in many things.

From Sidereel to SocialText.
From Scrum to Kanban.
From San Francisco to Palo Alto.
From Entertainment to Collaboration.
From External Facing to Intra-Facing.
From Ruby on Rails to a hybrid of technologies including an old friend, Perl!

It's the latter aspect that has given me cause and curiosity to look into and learn about an interesting testing framework. That framework is called Sikuli, and a "visual language" called Sikuli Script.

Sikuli started as a project at MIT. It's now an open source tool that can be used with a number of different applications. It can be used with the web, it can be used with Flash apps, it can be used with compiled applications on a number of different platforms. Yeah, OK, that's all fine and good, but aren't there plenty of tools out there that already fit that space? Why use another one?

I thought much the same thing, until I thought about a few of the applications I've wanted to poke around with in the past... some of them just aren't designed with test-ability in mind, or to be more charitable, some programs just bedevil the expectations of many tools currently available. Some apps don't have much in the way of an open API to access and help with the process. Selenium works great when you can access the object layer. What do you do when you can't access those attributes so easily? What if the front end is all you have, and all you are going to get? It's here that Sikuli starts to get interesting.

Sikuli can be run in a number of different ways, and the most likely way it will be run from a first timer's perspective is to use its IDE. The IDE puts front and center a number of function calls and simple tools that the user can take advantage of to make scripts. These scripts are similar in a lot of ways to Selenium/WebDriver and Capybara in what they do, with an interesting difference. The function calls can take images as their arguments. Not paths to images, actual images on the screen. Here's a simple example using a Win32 application I use with my scout troop called Troopmaster. The following is a very basic screenshot and just a couple of commands.

This admittedly trivial example performs a very simple set of instructions:

Load an app (Troopmaster).
Wait for a value on the screen to appear.
Open the Merit Badge Counselor tool.
Click on the "Add New" button (so we can create a new Merit Badge Counselor).

That's it! All very straightforward, all very basic stuff. The cool thing is that, in many ways, doing little more than these kind of steps, you can accomplish a lot of tasks. Sikuli uses image recognition to help determine where you want to do certain things. Based on those images, often even a non programmer or tester can put together tests or automate tasks to help them accomplish certain goals (it should be noted that Sikuli is not positioned solely as a testing framework. It's also used as a sort of "macro language" to help with automating basic repetitive tasks).

Now, of course, there's a lot more Sikuli can do and there are some frustrations and challenges that need more than just  "point here, click this, fill this in, Click OK". While very basic and trivial tasks can be done without extensive programming knowledge, to get beyond the training wheels basics, it helps to know what the architecture and language structure is. Sikuli is written in Jython, which is a Python implementation that allows the user to import and access many Java library functions, as well as to compile the source code down to a JVM. The user also has the full breadth of the Python language. If I ever wanted to have a good excuse to spend time with Python and get familiar with more than the basics, here's a great opportunity to do exactly that.

Again, this post is not meant to be an all encompassing tutorial. I installed it on my PC last week to get familiar with it, and wanted to get some first impressions out there. So where do I go from here? I want to get more familiar with doing things that are non trivial, and the best way I know how to do that is to, well, publicly declare that I'm going to do it. Does this sound like a new set of entries for the Practicum page? Hey! That's what it sounds like to me, too! Thus, it's time for another "bold boast"... let's see what we can do with Sikuli, and what we can't. Let's see in real time if it's a workable tool or if it's another "interesting, but..." kind of a framework. Also, let's give me a good excuse to poke around with and dive deeper into Python, let alone Jython and how it adds to this mix (to be frank, I'd never even heard of Jython before I downloaded this app, so I know almost nothing about it or its inner workings).

If you'd like to play along at home with me, you can get Sikuli and learn more about it at http://www.sikuli.org/