A Very Quick and Somewhat Incomplete History of Accessibility: Training for Accessibility: A Series (Part 2)

As I've been thinking about Accessibility and how I might construct training for it, I figured the History of Accessibility, at least as it relates to the United States, would be interesting. As the digital world has been with us since the end of World War II, many of the adaptations that are discussed came about after that, specifically starting with the 1960s.  

In the 1960s, researchers began exploring accessibility for people with disabilities. Early efforts focused on creating hardware and software adaptations for individuals with limited mobility or vision.

In 1973, the U.S. passed the Rehabilitation Act and specifically Section 504 was passed. This was an important step in that federally funded programs could not discriminate based on disability. Anyone who knows anything about federal programs gets that they can have a tremendous impact on software development, even back in the 70s. We see this today with the fact that lots of software get purchased by the federal government and they can often demand that certain requirements be met and companies will jump to make sure they are able to get those dollars. In this case, it made for a marketplace where accessibility technology had a chance to make money. 

The 1970s and 1980s were a time where Accessibility products and projects made a big jump. IBM introduced ScreenReader in the mid-1970s. While only available for IBM computers, it was still a working example where on-screen text could be reliably converted into recognizable (albeit synthesized) speech.  Apple OutSpoken provided a similar product for the Apple II, providing access to text-based applications.

The Refreshable Braille Display was another big jump, allowing users to type in to their computers and then read back the responses or actions from a flat display that would raise and lower small nubs that, when the user passed their fingers over them, would represent braille and allow blind or sight impaired users to "see" through touch, and moving to the next line would refresh all of the characters.  

A variety of alternate input devices became commonplace in the 1980s. An example I remember seeing early on in my late teens was a large trackball where the spinning ball was the size of a softball with large buttons. Sip-and-puff switches allowed users the ability to control devices and access switches by inhaling and exhaling.

In addition to the refreshable Braille Display, braille translation software made it possible to print off documents that were translated to braille.

A big step for Accessibility came with the popularization of the World Wide Web and other countries also stepping in and making laws that fought against discrimination of disabilities, as well as providing incentives for developing standards to help people with disabilities get access to information.

In the U.S. the Americans with Disabilities Act (ADA) of 1990 was passed and expanded on the Rehabilititation Act of 1973, though this was more of a physical infrastructure focus. In 1998, Section 504 got some additional strength with the passage of the ADA and Section 508, which specifically dealt with requiring federal agencies to comply with regulations making information and technology more readily available to people with disabilities. 

The UK passed the Disability Discrimination Act (DDA) in 1995. The EU likewise took center stage with the Web Accessibility Initiative (WAI) in 1997, sponsored by the World Wide Web Consortium (W3C). Australia introduced the Disability Discrimination Act (DDA) in 1992. 

The EU took steps to establish the World Wide Web Consortium (W3C), an international standards organization. It created the Web Content Accessibility Guidelines (WCAG) 1.0 in 1999, providing a framework for creating accessible websites. Today, most people who look at and consider Accessibility look at the WCAG requirements first, because if the WGAG requirements are met, a large percentage of any other country's Accessibility laws are included there. The WGAG standard continues to be debated and refined, and additional coverage has been added over the years. WCAG 2.1 is the current fully supported version but WCAG 2.2 and later versions have been in the works for quite some time.  

As time goes on and interaction with products becomes more defined by portability and touch, more accessibility features are being developed and are coming for mobile devices as well. Speech recognition, which has been an add-on for most home computer systems, is built in with most modern cell phones. Pinch to zoom and resizing are also tools that are accessible if not specifically designed for the purpose. Responsive Design is also a later development that helps considerably with accessibility. I've long said that the benefits one gets from examining agents and resizing for the display available get us 80% of the way to more accessible designs, as many of the elements needed to display the content also help with formatting data in a more accessible manner.

This is just the tip of an iceberg that could go on for a. considerable amount but this paints with a broad brush and gives some of the significant developments. I'm sure I missed several. If you feel there are some additional items or milestones I should have included, please let me know :).

Training for Accessibility: What Would I Say and How Would I Say It? A Series (Part 1)

Without going into details, I had a conversation about the possibility of doing training related to Accessibility and Inclusive Design. I've given talks about these topics and I've delivered workshops on them as well but I've always done so from the perspective of a software tester. Granted, that provides a lot of focus on advocacy but more times than not, it really comes down to "Here's what Accessibility is, here's why it's important, and here is how you can test for it."

A simple word cloud with some aspects just from this article.

I am realizing that there is a much bigger conversation we could be having here and with many more people. Since I keep saying I want to focus on Accessibility going forward, maybe I should put my money where my mouth is and go on record with some things. Maybe this could become the basis for a book, a training course, or some other set of strategies that we could use and leverage. Maybe this will give me a chance to talk out some ideas while I'm in that in-between phase of being gainfully employed and in what capacity. So if you will indulge me, I'm going to embark on a series of articles surrounding my thoughts on what I would say if I were to be your Accessibility Coach and Trainer. 

Ready? Let's go!

Defining Accessibility

Every first module in training tends to start with a definition so that we can all be talking from the same place and with the same ideas. To that end, I have historically focused on digital Accessibility but of course, true Accessibility goes well beyond the digital realm.

If we are serious about delivering Accessibility, we should start with the premise that Accessibility means we (collectively) work to create a world and an environment where everyone, regardless of their visual, auditory, cognitive, or mobility abilities, can fully participate in and enjoy all aspects of a meaningful and purposeful life. A bit heavy? Maybe but work with me here. Are we looking to say that we wish any less for those of us who are not blessed with the genetic lottery or have through no fault of our own had to deal with a physical or mental impairment? We live in a world where, too often, the normative folks get to have all the perks of their situation, while those who have disabilities often get second-level consideration, if they receive any consideration at all. 

Accessibility simply means we need to work to remove barriers that prevent people with disabilities from interacting with and enjoying the opportunities that life offers us. The first step in doing that is understanding that there is a divide between those who are fully able-bodied (or the phrase I like to use, "normative") and those who need some adaptations to participate at the same levels. What is seen as normative is often the path of least resistance. If any effort due to physical issues has to go into doing something, then that may often be the first line of focus to see what and why that is the case. Also, let's not kid ourselves, normative is often crowd-sourced and broadly agreed upon. Over time, disabilities get focused on or dismissed because enough people pay attention to them and it becomes an everyday part of life. 

If the community of Crossfit enthusiasts was to, for example, become the litmus test for the normative, we would see a lot more people suddenly identifying with and feeling as though they were dealing with disabilities. Seems a stretch, right? No pun intended here. My point is that normative is what people agree it is and the broader the population agrees with that, the more likely it will just be a standard part of life for those people who qualify. 

Normative people don't have to think about if the world is built for them. 

It just is, by default. 

For those who have a significant disability (whether it be a chronic and permanent situation or one in which we find ourselves temporarily inhabiting) we start to see the world differently.

Focusing on Accessibility means that we aim to identify barriers that could or would prevent people from accessing and enjoying the experiences and opportunities that others have, and then work to remove or at least mitigate those barriers.

In terms of the physical world, we see accommodations such as ramps leading into buildings that allow wheelchairs or walkers the ability to move effectively. We see and feel braille in items such as check-out kiosks and ATM machines, crossing signals, etc. We see closed captioning for the hearing impaired. All of these are accommodations that we have come to expect and consider part of everyday life but many places and organizations are not set up for this. In many ways, that is understandable. It would be tremendously expensive to retrofit every home to be accessible for everyone but we don't hesitate to make those accommodations when we are the ones that have to use them ourselves. 

I had the chance to experience this firsthand back in 2011 and again in 2013 due to a severe tibia break that had me unable to walk for an extended period. My house became a nightmare to navigate. My upstairs area was effectively off-limits to me for six weeks. On the occasion I needed to go upstairs, I had to literally sit on each stair and hoist myself via triceps extension, and shuffling to get to each stair. I recognized that, was I to have been in this position for a more permanent reason, the house would have to be retrofitted with a stair lift of some kind, or I'd have to accept the fact that I would lose access to anything happening upstairs in a meaningful way. 

Why did I walk you through that? I did that because at times that's what it takes for us to come to grips with the fact that what works for us one day may not work for us another and at some point what we took for granted as an everyday experience may not be available to us AT ALL at some point. In physical spaces, this can be a real challenge. In digital spaces, we have a lot more flexibility in the nature of how products are designed. The barriers to accommodation and accessibility are much lower.

At the digital product level, accessibility means we take into account the various ways in which people can, or can't, interact with devices like computers (and applications, websites, etc.), smartphones, and the media which is produced for each of those. Think of people who are completely blind or have any number of reduced vision issues. Think of those who are deaf or have hearing loss. Think of people who have moderate to profound mobility issues, everything from rheumatoid arthritis to full limb paralysis or absence. Also, there are a variety of cognitive challenges people can face and they can especially become apparent as people age. 

One key area people often miss when talking about accessibility is that it is too often framed around people with chronic or persistent disabilities. Yes, it is absolutely important we consider them in our design choices. It would be in my mind literally immoral not to. However, accessibility often benefits completely normative users in a variety of mundane situations. Have you ever been to a concert and received a phone call? Hard to talk with a sound system at full volume. Also, kind of hard to take that call if you have no real effective means to move to a quieter place. Here's where the ability of your phone to handle texts is not just a change of application but it's also an accessibility hack. "Oh, you can't hear me? I understand. Then let me text you instead." Accessibility in action :).

It can be as simple as TikTok including captioning by default or making it so that captioning is available. Many times I have found myself in situations where I am seeing a video but I am not in an environment where I can readily hear what is being said. With captioning, I can work with that and read what the person is saying without having to hear their voice.

Here's my quick and dirty definition of and the importance of Accessibility, what it is, and why we might want to care about it. Next time, I'll go into a little more depth on the history of Accessibility, how we got where we are, and how we might go forward from here.

I Guess Nothing Lasts Forever: TESTHEAD At Large

I've recently found myself in an interesting position - after working for the same company for the past decade, I'm now looking for a new job. To be clear, this isn't entirely by choice. However, I have no ill will or hard feelings for the company that has chosen to put me at liberty. They are making choices that make sense for them and I've been here before.  Granted, it's been twenty years since I had to deal with this in such a stark way but this brings me to a realization. For the first time in a decade, I am free to explore and consider whatever career I want. I am literally unsupervised. To borrow from the old joke, yeah, it freaks me out a little bit, too, but the possibilities are endless.

My Motto for today. BTW, this shirt with this motto is available at:
https://www.etsy.com/listing/671632845/i-am-currently-unsupervised-i-know-it

I remember talking many times with people and they asked me if I had the chance and the choice to go into exactly the line of work and the area that I wanted to, what would it be? For anyone who has followed this blog for any length of time, that might seem obvious.

I would like to actively explore and advocate for better accessibility and Inclusive Design, whether that be in the digital or the physical world.

What is interesting to me is the fact that when I started working with my previous company, accessibility was the first major project I was responsible for and worked towards. It developed in me a desire for advocacy and speaking about the topic for the better part of a decade. However, due to shifting needs, I haven't worked with a hands-on active work project around accessibility since 2018. I miss being actively engaged with this at a level beyond speaking about and writing about it. 

Over the years, I've seen firsthand how important it is to design and build software that is accessible to everyone, regardless of their abilities. I've come to realize that accessibility isn't just something that's nice to have - it's a fundamental aspect of good design. It's good business and frankly, it's something every one of us will have to come to grips with at some point in some capacity.

Thuis to that end, I have decided to come back to my old friend, TESTHEAD, and recommit to sharing accessibility ideas, approaches, methodologies, and hey, maybe dive deeper into some programming aspects and ways to make accessibility tools that myself and others might want to use.

I'm excited to explore new opportunities, and if a good one comes along that's not specifically focused on accessibility, I'll certainly not dismiss it. However, this is a chance to put that very specific feeler out there, to see if someone out there would be interested in a passionate accessibility advocate and having them join their team or even working peripherally with them. Regardless, this blog has been quiet for too long outside of live blogging of conferences. I hope you will join me in my journey to change that. 

Low-level Approaches for Testing AI/ML: an #InflectraCON2013 Live Blog

 One of the great parts of conferences like this is that I meet people I have interacted with for years. Jeroen is one of those people. We worked together on the book "How to Reduce the Cost of Software Testing" back in 2010 but we have never met in person before this week. We've had some great conversations here and now I finally see him present.

Jeroen Rosink

Sr. Test consultant, Squerist

I think it's safe to say everyone has been hit with some form of AI and ML in some capacity. If you need an explanation of AI and Machine learning, I'll let Chat GPT tell you ;).

AI, or Artificial Intelligence, refers to the development of computer systems that can perform tasks that would typically require human intelligence. These tasks might include things like recognizing speech or images, understanding natural language, making decisions, and solving problems. AI can be classified into various categories such as supervised learning, unsupervised learning, reinforcement learning, and deep learning.

Machine learning is a subset of AI that focuses on teaching computers how to learn from data without being explicitly programmed. In other words, it's a method of training algorithms to make predictions or decisions based on patterns in data. Machine learning algorithms can be trained on a variety of data types, including structured data (like spreadsheets) and unstructured data (like text or images). The most commonly used machine learning algorithms are supervised and unsupervised learning algorithms.

I mean, that's not bad, I'll take it. So I used AI to explain AI. What Inception level is this ;).

AI is always learning and it has been trained on large data sets. I often look at AI as a good research assistant. It can do some pretty good first-level drafting but it may miss out on some of the nuances and it may also not be completely up to date with the information it provides. Also, Machine Learning really comes down to ranking agents and probability. The more successes it establishes, the higher it ranks certain responses. To be clear, even with how rad AI and ML seem to be, we are still in the early days of it. We can have all sorts of debates as to how much AI will take over our work lives and make us obsolete. Personally, I don't think we are anywhere near that level but I'd be a fool to not pay attention to its advances. Therefore, we need to consider not just how we are going to deal with these things but how we are going to test them going forward.

 Jeroen talks about the confusion matrix and how that is used to test ML.

The confusion matrix is used to evaluate machine learning models, particularly in classification tasks. Think of it as a table with a number of correct and incorrect predictions made by a model for each class in a set of data.

The four possible outcomes are:
- true positives (TP)
- false positives (FP)
- true negatives (TN)
- false negatives (FN).

A true positive occurs when the model correctly predicts a positive instance.
A false positive occurs when the model incorrectly predicts a positive instance.
A true negative occurs when the model correctly predicts a negative instance.
A false negative occurs when the model incorrectly predicts a negative instance.

Jeroen has two approaches that he is recommending:

The Auditor's Approach

First, we perform a walkthrough so that we can see if the data is reliable and useful. From there, we do a Management Test to use data in enough volume to see if the data as presented works with small and larger numbers. If we can see that the data is relevant with one, and with 25, then we can see if it's relevant with 50 or 100, or 1000 and so on. We can't predict the output but we can have some suppositions as to what they might do.

The Blackhole Approach

This is an interesting approach in which we don't necessarily know what the data is or what we would actually have as data. We can't describe what is actually inside the black hole but we can describe what surrounds or is visible around the black hole. In this capacity, we look for patterns and anomalies that don't correspond with our expectations. If we see a pattern that doesn't match what we expect, we may have an issue or something that we should investigate but we are not 100% sure of that fact. Jeroen explained that there's a technique that can be used in the classic illustrations for "Where's Waldo?" The idea is that with a pen and making some marks on the page, we can figure out where Waldo is in about ten passes. To be clear, the system doesn't know where Waldo is, but it examines patterns in the image and breaks down the patterns to figure out where the item it is looking for might be.

These are neat ideas and frankly, I would not have considered these prior to today but be sure I'm going to think a lot more about these going forward :).

Technical Debt Is Not Free: an #InflectraCON2013 Live Blog

 

Chad Green

Director of Architecture, Glennis Solutions

When you hear the term "Technical Debt", what does that mean to you? Often the term "a quick and dirty fix" is used with the idea that it will be taken care of later. In short, anything you have to revisit later because of the limitations of today is specifically technical debt. 

The fact is, many of us have probably participated in the process of developing technical debt, whether we intended it or not. Even mature teams find themselves in technical debt, sometimes by active means, and sometimes by inaction. I remember working with an organization that had a great automation framework and it was very robust, with a lot of code libraries to support it. It was a great system, as long as the developers that created it were there to maintain and support it. However, we came to a point where our developers that did support this were no longer there and the team members that were there did not have the level of expertise necessary to maintain it. What was once a vital linchpin of our efforts became outdated and in some ways dangerous to do any work on. We went from having a reliable system to a need to modernize. In short, we woke up with a technical debt through no intentional effort on our part but we had to address it. We ultimately did but it took time and effort and a lot of iteration. We learned from that experience that we needed to make sure that what we developed had a broad base of support so that any of us could work on and maintain it.

Going into debt is not a crime or even a bad situation by itself. It can certainly become a bad situation if it's not addressed or worse, ignored. In many ways, we have to be more careful and make sure we are doing things in ways that are maintainable and understood. I went through this recently with some changes I proposed to a system that had a different way of handling API data (this came from suggestions of one of our devs). When I submitted it, the comments back were, "This is interesting and a way that we hadn't considered. We're not saying "no" but we may want to make sure we understand why we'd want to do it this way. Can we revisit this in the next sprint?" That's a perfectly reasonable request. Let's understand what making this change might be and how it might modify our testing methodology.

Technical debt sounds scary but there are ways to limit it or mitigate it. Take the time to do actual code reviews and make sure that the changes being made are understood. Talk about these things in Stand Up if necessary but surface these issues, as well as discuss these in your sprint reviews. Much of the time technical debt will be hiding in plain sight. The best way to deal with technical debt is to look for it and identify it as early as possible.

Bad Tests Running Wild - an #InflectraCON2023 Live Blog

 

Paul Grizzaffi

Senior QE Automation Architect, Vaco

Paul and I go way back. It's always fun to see my fellow in heavy metal arms at these events. We frequently talk music as much as we talk testing, so we are often in each other's sessions and today is no exception. Plus, Paul and I love making musical puns in our talk titles, and seeing Bad Tests Running Wild, I knew that was a reference to Scorpions' lead-off track from 1984's "Love at First Sting", aka "Bad Boys Running Wild"... yeah, this is going to be fun :).

The point here is that, especially with CI/CD pipelines, we need to have the tests pass to successfully complete and deploy an application. If a test fails, the whole process fails. By virtue of how tests run in a CI/CD pipeline, we need to make sure that any test that we have can run all the time, independent of any other test, and independent of any state of our product. This means a flakey test can really derail us. Note, this is not talking about a test legitimately failing or finding a fault. This is more the "random timeout because of a latency that occurs and that has nothing to do with our application".    

Let's think about how we create our calls and procedures. Do we have everything under our own umbrella? How much of our solution uses third-party code? Do we understand that third-party code? If we are using threaded processes for concurrency, are all of our components able to use those concurrent thread approaches?  

Let's think about configuration and how we set things up. Why do we want or need parallelization? Overall, it comes down to time and speed. I remember well our earlier setup with Jenkins from about a decade ago. It took us several hours to run everything in serial. Thus, we needed to set up the environment in such a  way that we could run four servers in parallel. At a point, we have to look at the costs of running our CI/CD pipeline vs. the time it takes to deploy. Our sweet spot was determined to be four servers running in parallel. Those four servers ran our tests in twenty minutes and then did our deployment if everything went smoothly. Going from several hours to twenty minutes was a big time saving but yes, it cost to set up robust enough servers to get those savings in time. After those four servers, we determined that adding more servers created a less favorable cost to time savings, as compared to running four servers. Still, it was critical to make sure that any tests we ran and any states that changed had to be all self-contained. No test was allowed to leave any residual footprints. Additionally, we had to ensure that our main server and out client machines were responding quickly enough to make sure that we didn't have potential latency with multiple machines (heck, spinning up a machine in a different server farm could mess everything up, so you needed to make sure that everything was proximate to each other.   

Also, we are only considering what happens when a test fails when we don't want it to or it's not supposed to fail. However, we also have to consider the flip side, which is what happens if a test passes that shouldn't? That's the flip side of a flaky test. What if we have made a change but our test is too generic to capture the specific error that we have introduced? That means we may well have introduced a bug that we didn't or wouldn't catch. 

Risks are always going to be present and our goal as testers and automation specialists is that we want to look at the potential risks that are present. What is the basic risk we need to mitigate? What happens when we deploy our systems? Do we have the ability to back out of a change? What do we need to do to redeploy if necessary? If we deploy do we have an easy way to monitor what has gone in? Paul makes the point that, if a change is potentially expensive, then you probably need human eyes to watch and monitor the situation. If there's little cost or risk of failures, then it could be handled without a person looking over it. Regardless, you will need to have the ability to monitor and to that end, you need logs that tell you meaningful information. More to the point, you need to know where the logs are and that they are actually accessible.

As always, exciting, interesting, and great fod for thought. Thanks, Paul. Rock on!!!

The Dark Side of Test Automation: an #InflectraCON2023 Live Blog

 

Jan Jaap Cannegieter

Principal Consultant, Squerist

Jan starts out this talk with the idea from Nicolas Carr's book "The Glass Cage" that "the introduction of automation decreases the craftsmanship of the process that is automated". I've seen this myself a number of times. There's a typical strategy that I know all too well:

- explore an application or workflow

- figure out the repeatable paths and patterns

- run them multiple times, each time capturing a little more into scripts so I don't have to keep typing

- ultimately capture what I need to and make sure it passes.

The challenge with this is that, by the time I'm done with all this, unless a test breaks, that test will now effectively run forever (or every time we do a build) and honestly, I don't think about it any longer. The question I should be asking is, "If a test always passes, is it really telling us anything?" Of course, it tells me something if the test breaks. What it tells me varies. It may indicate that there's a problem but it also may indicate a frailty in my test that I hadn't considered. Fix it, tweak it, make it pass again, and then... what?  

I'm emphasizing this because Jan is. Just because a test is automated doesn't necessarily tell us how good the testing is, just that we can do it over and over again. Likewise, just because a test is automated, it doesn't really give us much indication as to the quality of the testing itself. Let me give an example from my own recent testing which revolves around APIs. On one hand, I am able to find a variety of ways to handle GET and POST commands but on the other, do I really know that what I am doing actually makes sense? I know I have a test or a series of tests but do I actually have tests that are worth running repeatedly? 

I appreciate the fact that automation does something important but it may not be the importance we really want. Automation makes test efforts visible. It's hard to quantify exploratory sessions in a way that is easy to understand. By comparison, it's easy to quantify the statement, "I automated twenty tests this week". Still, much of the time, the energy I put into test automation saves me repetitive typing, so that part is great but it doesn't specifically find bugs for me or uncover other paths that I hadn't considered. 

There are five common misunderstandings when it comes down to test automation:

- the wish to automate everything

I have been in this situation a number of times and it typically becomes frustrating. More times than not, I find that I'm spending more time futzing with tooling than I am actually learning about or understanding the product. There's certainly a variety of benefits that come with automation but thinking the machines will make the testing more effective and frequent often misses the mark.

- you can save money with test automation

Anyone who has ever spent money on cloud infrastructure or on CI/CD pipelines realizes that often having more automated testing doesn't save money at all, it actually increases cycles and spending. Don't get me wrong, that may very well be valuable and helpful in the long run but thinking that automation is going to ultimately save money is short-sighted and in the short term, it absolutely will not save money. At best, it will preserve your investment... which in many cases is the same thing as saving money, just not in raw dollar terms.

- automation makes testing more accessible

Again, automation makes testing more "Visible" and "Quantifiable" but I'd argue that it's not really putting testing into more people's hands or making them more capable. It does allow the user who maintains pipelines to be able to wrap their heads around the coverage that exists but is it really adding to better testing? Subjective at best but definitely a backstop to help with regressions.

- every tester should learn how to program

I'd argue that every tester who ever takes a series of commands, saves them in a script, and then types one command instead of ten is programming. It's almost impossible not to. Granted, your programming may be in the guise of the shell but it is still programming. Add variables and parameters and you are de facto programming. From there, stepping into an IDE has a bit more learning but it's not a radical step. In other words, it's not a matter of, "Does every tester need to learn how to program?" We invariably will. To what level and at what depth is the broader question.

 

- automation = tooling

I'm going to argue that this is both a "yes" and "no". As I said previously, you can do a lot of test automation using nothing but a bash shell (and I have lots of scripts that prove this point). Still, how do scripts work? They work by calling commands that pipe the output to some other command and then based on what we pipe to what, we do one thing or we do something else. Is this classic test tooling as we are used to thinking about it? No. Is it test tooling? Well, yes. Still, I think if you were to present this to a traditional developer, they would maybe raise an eyebrow if you explain this as test tooling. 

My rationale and it seems Jan feels a similar way is that we need to look at automated testing as more than just a technical problem. There are organizational concerns, there are perception issues, and there are communication issues. Having automation in place is not sufficient. We need to have a clear understanding of what automation is providing. We need clarity on what we are actually testing. We need to have an understanding of how robust our testing actually is and also how much of our testing is tangibly capturable in an automated test. What does our testing actually cover? How much does it cover? What does running one test tell us versus what ten tests tell us? Are we really learning more with the ten tests we run or is it just a number to show we have lots of tests?

The real answer to this comes down to, "Why are we testing in the first place?" We hope to get the information we can make judgment calls on and ultimately, automated tests have a limited ability to make judgment calls (if they can make them at all). People need to analyze and consider to see what is going on and if it is actually worthwhile. It has its place, to be sure, and I wouldn't want my CI/CD environments running without them but let's not confuse having a lot of tests with having good tests.

Castle Defense 101 (aka Threat Modeling): an #InflectraCON2023 Live Blog

Well, good morning to everyone. We woke up to some excitement today. A fore alarm went off at 7:30 a.m. this morning, so I had to evacuate the venue. It was interesting watching me react to what was happening. I've always been fond of saying I'd be one to just get up and get out but as it turns out, no, I had the presence of mind to gather my stuff quickly and push it into some bags, grab my laptop and backpack, and then walked out. Fortunately, I travel relatively lean as a principle so it didn't take me long but were it a more dire situation, I fear I may have been one of those stragglers that might have been potentially cut off. All's well that ends well but yeah, gave me pause to think, to say the least.

Anyway, on to today's sessions.

Gene Gotimer

DevSecOps Engineer, Praeses

This talk on Castle Defense is both entertaining and an interesting look at what we might face as potential security issues and how we want to protect against potential attacks. When we think of castles, we often think of grand, large structures. Interestingly, castles all start as smaller strutcures. If we think of the old chessboards and the character of the Rook (or the Castle, yes), they are minimalist towers in many cases and typically, that's where keep and bailey castles start, too. They may be small or not terribly imposing but they can still be remarkably effective at defense if set up correctly. This is an interesting metaphor when it comes to security because, in many ways, security is graded on a curve. It also helps to know what your castle or fortress is meant to defend. A manor home for a noble is designed to keep people out. Prisons are meant to keep people in. It's important to know what you are protecting and what direction matters. 

In addition to the conversations about potential security threats for applications, we are taking the time to look at and analyze actual castles from the medieval period and later to see what they did and how they set up their defenses, and then analyze ways we could undermine its security. Granted, some of these castles have been modernized and no longer set up in a logical way that would have addressed the threats of the past (an example of a castle in the Netherlands shows a ground-level manor with what looks to be no defensive walls, windows down to the ground, and a river flowing by outside. In short, it doesn't look to be defensive in any meaningful way, until you see the center tower. That tower resembles what may have been the original structure, with a broad overhand with machicolations (I love that word so much (LOL!) ) but you can see that time and necessities have changed how the building is used. It's original threat modeling from when it was built wasn't necessary for later centuries, so the building was adapted to face more modern realities. Many castle fortifications made sense in the era of catapult and trebuchet but became obsolete with the advent of gunpowder and cannons.

 So let's consider this in the modern world. We aren't building castles and keeps in the literal sense (generally speaking for this audience) but our applications are in many ways our castles. If they are breached or hacked, our data, our financial well-being, and our reputations are on the line, so the threats, while different, are every bit as potentially devastating. Thus we need to put time and attention towards making a rational and logical level of security for our applications. Some situations are going to require more hardening than others. If you have an informational site with no database backend for transactions, your threat modeling is going to be smaller and less intensive compared to a site that handles the personal data of individuals or the literal handling of payments. A WordPress blog is going to be lower in priority compared to a banking app. We need to measure our time and investments for the threats that make sense.

There's a site called "Threat Modeling Manifesto" that spells out a broad range of these possible attacks and threats and how to handle them. From their headline:

Threat modeling is analyzing representations of a system to highlight concerns about security and privacy characteristics.

At the highest levels, when we threat model, we ask four key questions:

• What are we working on?
• What can go wrong?
• What are we going to do about it?
• Did we do a good enough job?

This was an interesting way to talk about this topic and I applaud the creative approach. It took a potntially dry topic and made it a lot more engaging.

Being an A11y: Why Accessibility Advocacy Matters: my talk from #InflectraCON2023

Accessibility is a broad area. It can be applied to many different scenarios and can be met in many different ways. At the end of the day, though, we are dealing with people with challenges and concerns that, let's face it, most if not all of us will face if we live long enough. 

Accessibility is more than checking off a box that says "We are compliant". It is advocating for people to be able to effectively participate in daily life as any of us would, with accommodations where necessary. 

In this talk, I will show you areas where we can do better to make products more usable, not just for those with physical disabilities but for all users. I will demonstrate tools and techniques to help test as well as make a case on behalf of those people who are not able to speak for themselves.

Michael Larsen

Senior Quality Assurance Engineer, Learning Technologies Group/PeopleFluent 

The key to this talk this go around was that I stepped a bit away from the what and the how (still important) and emphasize the "why". This was less a talk about tools and processes (though I touched on them) and instead emphasized ways we could advocate for Accessibility. As always, I owe a big round of thanks to Jeremy Sydik (his 10 principles will probably always be a keystone to my presentations) and to Albert Gareev (the HUMBLE principles are still, in my mind, the easiest way to encourage Accessibility advocacy regardless of your skill level).

I do want to share this tweet the organizers of InflectaCON shared because, wow, this made my day :).

“Think about the future - you may end up benefiting from your own design.”

The presentation "Being an A11y: Why Accessibility Advocacy Matters" by @mkltesthead was a #TeamInflectra favorite and rated 10/10❗#InflectraCon pic.twitter.com/ToamByL7ek

— InflectraCON (@InflectraCon) April 20, 2023

Accelerating Quality with Conscious Deliveries: an #InflectraCON2013 Live Blog

So this is fun. Lalit and I have known each other for years. We have attended conferences together. I've been interviewed by Lalit for Tea Time With Testers. We've worked together within the Association for Software Testing on a variety of initiatives. Having said all that, I think this is the first time I've actually seen/heard Lalit speak :).

Lalitkumar Bhamare

Tech Arch Manager, Accenture Song

It's interesting to realize that with all of the technological advances we have had over the past thirty years (probably longer but I've only been in the game for three decades), we still have to pick from the speed, cost, and quality triangle (you know it, "Fast, Cheap, Good. Pick two!"). It seems that if any shift is going to happen, it typically happens at the "Good" part, meaning that if any pressure comes into the situation, the quality side is the side that ends up bending. Granted, often that means we get "good enough" and for many people, that is sufficient. 

The irony is that we don't have to settle for good enough but it will require that upfront planning and resources be allocated to make sure that quality is reinforced. This comes down to requiring people to be motivated to provide not just good testing but a mindset of the importance of testing beyond the busywork of automation and declaring that testing has been performed. 

We had a little discussion about apps we actually like using. Recently due to life circumstances, I've become more familiar with healthcare apps. My company's insurance provider has done big on telemedicine recently and to that extent, it seems they have either found out I'm a tester or I just tick off a lot of boxes for them because they have thrown just about every app and tool my direction to manage healthcare and treatment options from a digital perspective. Some of these apps have been really helpful and some of them have been... less than desirable, to say the least. To be clear, these apps are not developed by the insurance provider but they are either partnerships or investments that my insurance provider has made and encourages use. It's interesting to compare them and see what makes them "quality" products vs. not-so-good quality. Also, some apps have great quality in some areas while being less good in others. A perfect example of this is an app I'm involved with that focuses on weight management specifically for people who are at risk for Type 2 Diabetes (which family history points to me being, so I'm part of their initiative for that reason). In areas where real human interaction takes place, it's great. However, they have recently made decisions to limit website updates and almost exclusively manage via their mobile app. In one way, this makes sense, as we are more likely to be within arms reach of our phones at meal breaks as compared to our computers. Still, I type way faster on a computer than I do on my phone, so invariably, their change has resulted in my updates being delayed, sometimes by days, because it's less convenient for me to enter the details. I'm curious if anyone on their team even brought up this possibility.

Lalit emphasized three "P" areas of quality consideration. You have a Project aspect, a People aspect, and a Product aspect. He additionally emphasizes the 4 "E"s of quality. Enable, Engage, Execute, and Evaluate. The 4Es apply to each of the 3Ps. Granted, each of these elements has a context based on where it is applied and there are biases that come into play, it uses the story of the "Parable of the Elephant" where our limitations often constrain our vision and view of an aspect of something (I had a good laugh realizing that Lalit chose Dieter F. Uchdorf's telling of this story. It took me a minute but I kept thinking "wait, I know that voice" (LOL!) ).

Over time, we need to be open to learning new things and incorporating more knowledge of the potential for requirements, and translating that to concrete actions that can be used. As testers, we may be doing a lot of work but we may not actually be aligned with actual business issues or challenges. As I love to borrow from Steve Covey, how infuriating is it to know we've climbed a significantly tall ladder only to realize we've placed it against the wrong wall?

We know that we cannot engineer quality, at least not in a literal sense. What we can do is preserve as much of the product's intended integrity as possible and take steps to make sure that we are learning and focusing on areas to make sure that we are creating the best product we can. To that end, having quality experience sessions can help to inform how a product is being used and what can be done going forward. Ideally, these considerations are made early on in the life of the product or as it is being developed. For that to be effective, it requires people with a focus on quality to ask questions and experiment with requirements early on. The later this happens, the less likely they will be of value in design but it might be very demoralizing to realize the "right ladder, wrong wall" problem is happening after we've climbed quite a bit.

To wrap this up, if you need to have a simple thing to consider and practice, "test early, test small, and test continuously" is a pretty good approach, and apply it to all of the areas you interact with. if you find it valuable, share the approach and help it expand through the organization.