Managing The Test Data Nightmare with @AutomationPanda (@Xpanxion #QASummit 2021) : Live Blog

 Wooo hoooo! The second talk is done, I am officially free of obligations today :). However, conference still moves on and this session covers an area that I personally struggle with. My personal job does a lot of data transformation so I have a possibly endless range of test data that can be generated and transformed. 

Test data shows up everywhere. Not just the data needed to make the test work but your browser choice, your necessary created artifacts, before and after dependencies, etc.

Static Data is often created before testing. it's good fr slow or complicated data. It may make tests run faster but it may make tests brittle as data changes, and it may turn stale over time. Dynamic data gets created at run time for tests, it can avoid becoming brittle, it's exclusive tho that tests, creation and run time, it can slow down individual tests due to the creation dependencies, and it will need to be cleaned up after the test is finished.

The truth is, I probably initially do about 70% manually configured data and about 30% dynamic/automated data creation. The intermediate data that I create is always created dynamically (that's the nature of data transformation tests). Ultimately, my goal is to be able to take data from our database and dynamically generate flat files.

Additionally, there is a variety of test control inputs that we need to keep track of. Our browser. our destination URL, basically all information that can be entered and routed. There are also output references that we may know or we may not know anything about. 

Maximizing Success with Data Testing: Live at #STPCON

Regardless of what we develop, what platform, what use case, everything we do at some point comes down to data. Without data there's not much point to using any application.

Lots of applications require copious amounts of data, reliably accessible, reliably recreateable, and confirmed content that will meet the needs of our tests. At the same time, we may want to generate loads of fresh data to drive our applications, inform our decisions, or give us a sense of security that the data we create is safe and protected from prying eyes. Regardless of who you are and where you are in the application development cycle, data is vital, and its care, feeding and protection is critical.

Smita Mishra is giving us a run down on Big Data / Enterprise Data Warehouse / ETL Process (Extract, Transport, Load) and Business Intelligence practices. We can test with 1KB of data or 1TB of data. The principles of testing are the same, but the order of magnitude difference can be huge.

Big Data in my world view is used to describe really large sets, so large that it cannot easily fit into a standard database or file system. Smita points out that 5 Petabytes or more defines "Big Data". Smita also showed us an example of an "Internet Minute" and what happens and transmits during a typical minute over the Internet. Is anyone surprised that the largest bulk of data comes from Netflix ;)?

Big Data requires different approaches for storage and processing. Large parallel systems, databases of databases, distributed cloud system implementations, and large scale aggregation tools all come into play. Ultimately, it's just a broad array of tools designed to work together to cut massive data amounts to some manageable level.

In my own world, I have not yet had to get into really big data, but I do have to consider data that spans multiple machines and instances. Additionally, while I sometimes think Business Intelligence is an overused word and a bit flighty in its meaning, it really comes down to data mining, analytical processing, querying and reporting. That process itself is not too hard to wrap your head around, but again, the order of magnitude with Big Data applications makes it a more challenging endeavor. Order of operations, and aggregation/drill down becomes essential. Consider it a little bit like making Tradizionale Balsamic Vinegar, in the sense that your end product effectively gets moved to ever smaller barrels as the concentration level increases. I'll admit, that's a weird comparison, but in a sense it's apt. The Tradizionale process can't go backwards, and your data queries can't either.

There are some unique challenges related to data warehouse testing. Consistency and quality of data is problematic. Even in small sample sets, inconsistent data formats and missing values can mess up tests and deployments. Big Data takes those same issues and makes them writ large. we need to consider the entry points of our data, and to extend that Tradizionale example, each paring down step and aggregation entry point needs to verify consistency. If you find an error, stop that error at the point that you find it, and don't case it to be passed on and cascade through the system.

Retro Book Review: The Manga Guide to Databases

Since I’m in the process of doing Noah Sussman’s Technical Tester Challenge, and part of that involves setting up and maintaining a server with an actively running database, I figured now was as good a time as any to take a look at Mana Takahashi and Shoko Azuma's "The Manga Guide to Databases” as a refresher and, maybe, teach me a few new things.


I’m already a fan of "The Manga Guide to” series, so I figured that their take on databases would be in the same vein as their other titles (an accompanying storyline, an emphasis on practical topic coverage, and an emphasis on “kawaii”). To meet that end, we are introduced to Princess Raruna, heir apparent to the Kingdom of Cod. We also meet her attendant, Cain, and a fairy named Tico that teaches them about databases… and anyone familiar with Manga has not batted an eye with that kind of a description (and sure, if you looked at the cover, you could probably have figured that out as well ;) ). For those not already familiar with Manga and their tropes, this might seem a bit strange, but go with it. Seriously.


I can see some of you already thinking “OK, sure, I can see teaching about stars, or maybe even math using Manga, but databases? That’s a bit of a stretch, isn’t it?" Well, let’s take a closer look.


Chapter One introduces us to the Kindom of Kod’s main export… fruit (yeah, you were thinking fish. Everyone thinks fish, but no, it’s fruit). Through the bureaucratic and messy system that they have in place, the case is made for why a database is important in the first place, to reduce errors, keep track of important data, and to make sure that data isn’t duplicated in appropriately or not updated when and where it needs to be. This chapter also sets the stage with the scenarios and back story to help define how the database will need to be set up and managed.


Chapter Two takes the idea of a database further and discusses what relational databases are, and how they differ from other system such as hierarchical and networked databases. Fields and records are explained as vertical columns (attributes of a relationship) and records (individual collections of various attributes as relates to one given entity at a time). Tables hold these fields and records, and a variety of operations can be performed to both input and extract/format the data to be viewed.


Chapter Three goes into the process of designing a database, starting with creating an entity-relationship (E-R) model, and establishing the types of relationship an entity can have (one to one, one to many, many to many). From there, a table is designed, and by examining the relationships, we can see where data is duplicated. We can divide the big table into smaller, interrelated tables in a process called normalization. The concept of both primary and foreign keys are also introduced.


Chapter Four introduces us to the Structured Query Language, or SQL. SQL allows users to perform functions that allow them to define, operate, and control data. SELECT statement allow users to select specific fields to display and show values of those fields. The WHERE statement allows users to specify conditions as to what records are displayed. INSERT, UPDATE, and DELETE statements let users insert, update and delete data. CREATE TABLE lets a user create a new table, where DROP table lets a user remove (drop) an existing table.


Chapter Five focuses on how to operate a database, including how to set user privileges for a database, how to use locking and ensures consistency with multiple users, setting up indexes to perform faster searches, examining transactions and how they can be “rolled forward or rolled back”, and options for disaster recovery and database repair capabilities.


Chapter Six shows us how the proliferation of databases affects everyday things that we do, and that we are likely dealing with them in areas we otherwise would not consider (every site that this book review will appear on has a database to store them, and that’s at the simplest level. The chapter also shows us examples of distributed databases, database partitioning, two-phase commits, database replication and the use of stored procedures and triggers to perform commonly repeated tasks.


The book ends with a short Appendix with a summary of the most commonly used SQL commands (which would probably make for a nice little project for a dynamic allocation of commands for a web site example page.


Bottom Line:


If you are an old hand at using databases in general and SQL commands in particular, there’s probably not a whole lot of new material for you here. For those who are just getting into working with databases, this is a much more fun and straightforward way of teaching the ideas than I’ve seen, well, just about anywhere. Do note that this is not going to be the be all and end all of learning about databases, SQL queries or how to effectively design databases. It will, however, go a long way in giving those people who want to learn how to make or manage relational, SQL based databases a simple framework to hang future ideas and learning from.