Inclusive Meta Paradox Frameworks: A Little Shameless Self Promotion

I realize I'm terrible at promoting myself and the things that I'm doing. Having said that, I do want to encourage everyone to see what I'm up to and with that, I'm sharing a podcast I recorded with Mark Tomlinson for STPRadio.

Listen to "STPCON Spring 2019 Michael Larsen on Inclusive Meta Paradox Frameworks" on Spreaker.

You know the old saying "When the going gets weird, the weird turn pro?" Well, if you don't you do now :). Seriously, I love this title. Thank you, Mark, this is great.

Also, for those of you who are intimately familiar with my editing style on "The Testing Show", you may think that I am always smooth and flawless in my delivery, without any wasted breaths. Yep, it's true, no one on The Testing Show breathes... kidding, but now that I've planted that little seed in your head, I'll be the next time you listen to an episode you'll be subconsciously dwelling on that ;). My point is, Mark keeps it real and whatever was said as it was said is there in real time, so if you are curious as to how I really sound when I'm interviewed, here's your chance.

In this podcast, I talk about my workshop around "building a framework from scratch" (and yes after I finish this presentation I am going to start unpacking it and posting it here) as well as my talk on Accessibility and Inclusive Design and how they can be used to help Future Proof software.

If you will be at STPCon and you will be in my presentations, here's a taste of what to expect. If not, well, you get that anyway just by listening. Have fun and if you like the podcast, tell a friend about it, please.

Read Between the Lines: a #30DaysOfTesting Testability Entry

I feel like I'm on a roll here. Let's see if we can keep this 30 Days of Testing going :).


Get access to your source control and find the active branches for your application. Has anything changed that you didn’t consider?

Our software is hosted on our own GitLab server, so there are a variety of branches that we test with. We have been focusing on specific Sprint branches, so it makes it easy to see what was introduced in any given sprint. Since we have been focusing on a Responsive redesign of our product, there has been a significant push in changing the way that components are displayed in a variety of devices and user agents. Without getting into too many details, yes, there are a variety of updates that, by looking at the actual code commits, are in areas that I am testing but had changes that affect areas I hadn't specifically considered to look at.

The cool thing here is that I can see what the specific changes are and what components the changes will affect. With this knowledge, I can see if there are avenues I can use to do additional testing. Also, when I notice that some areas have been changed and ID's have been modified or added, it gives me additional areas to consider and see if there are similar areas that could be modified. Sorry to say that I can't really go into any additional depth because, as the usual sly comment entails "I could tell you, but then I'd have to kill you!" No, that's not true, but really, if I tell you what changes are being made I could certainly endanger my employment and, yeah, I'm not doing that.

Walking in Your Footsteps: a #30DaysOfTesting Testability Entry

OK, this gets me to a third of the way through the challenge. Fortunately, this is a relatively easy entry as I get to talk about other people doing great stuff and I'm glad to oblige. With that, let's keep this 30 Days of Testability Party going, shall we?

Follow and share three people on LinkedIn/ Twitter who regularly talk about testability.

Sweet, I get to brag up some people. Always fun.

1. Alan Richardson (Alan's LinkedIn, Twitter: @EvilTester)

If you read my previous entry about sharing a video, this name should be familiar. He also has a pretty extensive blog and a lot of learning resources via his own books and via YouTube.

2. Maria Kedemo  (Maria's LinkedIn,   Twitter: @mariakedemo)

Maria covers a lot of ground and has excellent content that she frequently shares. As to the topic at hand, check out her "Testability Awakens" article she wrote for Testing Trapeze.

3 Jim Hazen (Jim's LinkedIn,  Twitter: @JimHazen4u)

Jim has a book called "Before The Code: First Steps to Automation in Testing" that's worth a look. I highly suggest you give it a read.

That Voodoo That You Do: a #30DaysOfTestingTestability Entry

Hopefully, you are having some fun with me as I keep this going. This past week was a bit much leading into my daughter performing High School Musical (her last high school drama production) so there was a lot of activity around that so I'm going to be a bit aggressive with the remaining days of March so I can rightly finish the 30 Days of Testability challenge (or at least get as close to finishing on time as I possibly can).

With that...

In your next story kick off session ask, ‘how are we going to test this?’ Share the test ideas and techniques that are suggested.

This was a fun experiment in that I did get to do this for a specific feature that was getting a Responsive update. There were a variety of features that hadn't been looked at as thoroughly in recent efforts (even before the responsive updates) so I got to ask this question quite a bit. The good news is my Engineering team is totally cool with me asking these kinds of questions. We did some brainstorming and I made some suggestions of ways we could better examine how the application is working.

- some ideas revolve around adding calls to the API so that I can query and set values without having to fire up a browser and look at a bunch of things on the screen.

- some ideas revolve around actual configuration enhancerments of an application so that I can tweak various things while I'm setting up a test environment.

- some ideas revolver around making sure that elements are named well (whenever possible) and that duplication is minimal. Additionally with that, do anything we can to avoid dynamically generating elements. If that's not possible, let's at least make sure that we have a mechanism in place to easily discover those values and utilize them.

The biggest benefit I find with these conversations is that the development team and I can get on the same page much more quickly. In addition, it also allows me to see a bit of writing under the board and better understand the app a layer at a time. That's a good outcome for everyone, if you ask me :).

A Touch of Evil: a #30DaysOfTesting Testability Entry

Do any of you have any idea how long I have waited to be able to use Judas Priest's "Painkiller" ballad in a blog title? Way too long, that's how long ;). With much thanks to Alan Richardson (i.e @EvilTester) for so very often having just what I need right when I need it.


Share a video about testability with your testing peers at your company.

What is Testability vs Automatability? How to improve your Software Testing.
there are some very cool concepts in this video and hey, there's a supporting blog post to go with it. Yay Alan!!!

So what are the takeaways here?

Just because you can automate something doesn't necessarily means it's testable. Automatability means that something else can interact with it. Just because that is capable to be done doesn't necessarily mean that it's going to be a good experience for a user.

Alan's TL;DR drives this point home very effectively: "Testability is for humans. Automatability (Automatizability) is for applications."

Testability can add a number of options to a program and can help to make a system more enjoyable for a user. It may help with making an application more automatizable (great word, Alan, I'm adding that one to my lexicon) but it doesn't guarantee that it will.

In any event, the video and accompanying blog post are excellent. Check them out :).

Is it in the Playbook?: a #30DaysOfTesting Testability Entry

Here's another entry for the "30 Days of Testing" Testability challenge. Let's keep this ball rolling.

Find out if your application has an operational manual or runbook? How can you test this?

Put simply, yes, it does. More to the point, I've been one of a few people that has actually updated this documentation with new feature information as well as release notes. To that end, I absolutely have familiarity with what goes in it, how to put content in there and how to look for details about how our software works and what options are available to me.

Having said that, our software has a fairly high level of complexity and thus our documentation needs to reflect that. While I will not say that I look at every single help page, I do look at it from a macro level as I'm the one who builds those help files.

What I do try to do is look at the top level details and traverse them the way I imagine that an everyday user of our product might. What could this tell me?

First and foremost, it tells me what we are telling our customers as to how we should be doing something. That's an important avenue to look at and consider while I am testing because that's what we intend our customers to be focusing on doing to complete their workflows. How often do I do this and determine that I have a different or better way of performing a task? Surprisingly often, as I have come to see. Of course, part of that is the fact that I know the administrator interface and many times using that is *way* more effective than performing the in-app methods. However, since our customers do not have access to that level of interaction (well, most don't) I need to remind myself to either go with the published methods or make a case that, maybe, our backdoor methods probably need to be considered as a way to let our customers do things, too.

To borrow from Red and Overly Sarcastic Productions... "So... yeah!"

Frame By Frame: A #30DaysOfTesting Testability Entry

I'm going to have to get aggressive if I'm going to make the end of month deadline. therefore, I'm going to try to be more specific with these posts and discuss them in a way that will allow me to be more targeted and to post more entries. It's becoming a matter of pride now! Anyway, more "Thirty Days of Testability".

Explore the output of your applications analytics, how can they help guide your testing?

I'll take a two-pronged approach with this. I'll talk a bit aboiut my company's product and a bit about what I use on my own TESTHEAD blog.

With my company, we have two applications we use and monitor for Analytics. One is more for overall HTTP traffic and demographics (such as what pages get hit the most, what browsers are used the most and what time of days do people most use the application, as well as how many parallel connections are we maintaining at any given time. that helps me define what I should prioritize in my testing as well as what level of load and other parameters I might want to consider.

The second tool we use is based on feature analytics, as in "what features in our product are our customers actually using?" We've had a few times in the past where we had a request to get something implemented and that implementation was everything to getting the deal. From there, it often meant we would maintain and keep a feature running which was important for a small set of users but meant a big part of our business. Sometimes though, we would discover that an organization would demand something, later for us to discover that their adoption rate of that feature was low to non-existent. That often meant there would be follow-up discussions. Those discussions would help us to decide if the issue was worth us keeping the feature. Perhaps we weren't making a compelling enough story for why it would be important for the organization wanting to use it. Alternately, we also often decided that the feature had such low adoption that turning it off would have little overall effect on the use of the product. We would then gracefully deprecate that feature.

In short, getting familiar with your product's analytics can tell you a lot about what is being used but also what isn't.

I'm Gonna' Get Close To You - #30DaysOfTesting Testability

Since I don't want to put too many posts in one place, I'll use my catch-up time to mentions some other things going on in the TESTHEAD world while I do this. I don't want to be too gratuitous and self-promoting. Also, thanks to Queensryche to have a song all about monitoring to save my song title titles (say that ten times fast ;) ). OK, their version is about stalking but why quibble ;).

First off, there's a new episode of The Testing Show up over at Qualitest and on Apple Podcasts. This month Matt and I are talking with Lisa Crispin of mabl and Jessica Ingrassellino of SalesForce.org about Advanced Agile and DevOps and how those things are both similar and different. I think we did a good interview this month. Well, I almost always think we do good interviews but I especially enjoyed doing this one so I hope you will go and give it a listen.

Anyway, on with the "Thirty Days of Testability". As you might notice, today is the fifteenth. This is entry five.  Catchup in full effect. You have been warned ;).

What monitoring system is used for your application? Do the alerts it has configured reflect what you test?

Again, we are going to be considering Socialtext as it is currently implemented as a standalone product because that's all I can really talk about. Before this week, I had a partial understanding of what we actually do for this and some holes in that knowledge. I'm fortunate in that I have a couple of great Ops people who are associated with our team. I should also mention that Socialtext can be deployed in two manners. First is a hosted SAAS option and second is a local installation. We leave the monitoring of the local installations to the small percentage of our customers who prefer to do that. The majority of our customers utilize our SAAS option and therefore we host their servers. To that end, we use the following (I'm pretty sure I'm not spilling any trade secrets here, so this should be OK. If this post changes dramatically between when I post it and tomorrow, well, I'll have learned differently ;) ). Anyway:

for Monitoring the systems  (CPU, Disk space, https, http) we use a tool called Nagios.

For monitoring site uptime, https lookup, and time to respond, we use an external app called Alertra.

In addition to those two tools, we also have a variety of hand-rolled scripts that allow us to scan all of our servers looking for specific aspects of Socialtext instances and services to see if there are any issues that need attention. Examples here are things like the IP address, the hostname as viewed by the public and that it is accessible, that we are running certain key services (search, replication, cron, mail, ntp, our scheduler, our search implementation, what version that particular server is running, etc.).

The second part of the question deserves a legitimate answer and that is "Yes" and  "No". Yes, in that some of the alerts map to what we test but "no" in that there's a lot of areas we don't actively test as consistently as we should. The chat with our ops team was definitely enlightening and has given me some ideas of what I can do to improve on that front. What are those things? Well, other than verifying that we are actively doing things that affect and trigger those alerts, I will have to ask that you respect the fact that I am now veering into trade secret territory and I kinda' like the idea of keeping my job ;). 

Who You Gonna' Call? #30DaysOfTesting Testability

Wow, time flies when you are doing actual work and you are trying to get your talk and workshop done and rehearsed for STPCon (seriously if you are coming up to STPCon in Northern California first week of April, come join my talk and workshop or if I'm not your cup of tea at least say "hi" ;) ).

Anyway, on with the "Thirty Days of Testability".  As you might notice, today is the fifteenth. This is entry four. Yeah, I'm going to be playing catchup. You have been warned ;).


Do you know what your top three customer impacting issues are? How could you find out?

This is surprisingly easy for my immediate team. It's less so for my extended team. I have no idea who is coming in at what point here, so I'll say this for those who are new.


Currently, I work for Socialtext. Socialtext used to be a standalone product and for a number of customers, it still is. However, back in 2012, Socialtext was acquired by a company called PeopleFluent. PeopleFluent specializes in HR Tools as their name might indicate. PeopleFluent is also a much larger company by comparison (Socialtext as a standalone group is all of ten people). Last year, PeopleFluent was acquired by Learning Technology Group (LTG) located in the UK and with offices all over the world. Thus, when I have to think about my top three customer impacts, I have to ask clarifying questions. Are we talking about Socialtext? PeopleFluent? LTG? Interestingly enough, since Socialtext is the base platform that many of the PeopleFluent and LTG products run on and interact with, it's entirely possible that Socialtext will not even enter into the top issues of anyone outside of Socialtext and at other times a Socialtext issue can be the number 1 issue for everyone. Still with me? Excellent :).

So to keep things simple, I'll focus on Socialtext standalone and how I would determine what the biggest issues are for us. The simple answer is I can reach out to our secret agent in the field... ok, it's nowhere near that cool. We don't really have a secret agent but we do have a great customer engagement engineer and frankly, a lot of the time that's just as good. I can count on one hand the number of times when I have announced an update on our staging server (read: our personal production server) and not heard a reply from this person of "hey, what's new on staging?" They make it their responsibility to be clear and up to date with every single feature and every option available in the product. They also make a lot of sample accounts and customizations to our product to push the edges of what the product can actually do. If there is any question as to what is a pain point or an issue with a customer, any customer, they are my first point of contact. Sure, we have a CRM and a bug database but the majority of the time, if I really want to see what is happening and what's really important, I know who I am going to call... or bring up in a chat message. I mean come on, this is 2019 after all ;).

Sponsoring a "New" Book? #30DaysOfTesting Testability

Observations for when you start looking at various book titles related to niche topics in software testing. First is that they can be hard to find. Second, they can be mighty pricy. What is the cost when it comes to learning? True, but there are a variety of factors one has to consider, such as, "OK, I really don't have close to $100 to drop for a specialty book at this exact moment."

Having said that, occasionally one finds interesting things in interesting places and sometimes those things are in the process of percolating. It is in this guide that I have chosen to make my mark for "Day Three" of this challenge... yes, I'm writing about it on "Day Seven", let's not quibble here.


Begin reading a book related to testability and share your learnings by Day 30.

To this end, I've decided to help fund a book in the process of being written (or at least to be written for Leanpub distribution). Said book?

Team Guide to Software Testability

by Ash Winter and Rob Meaney

Learn practical insights on how testability can help bring team members together to observe, understand and control customer needs, ensuring fitness and predictability of deliveries.

Now I will have to say up front that I will probably not be able to provide a 100% complete read of this book because the book is literally being written as I've purchased it. However, I will be more than happy to review what has been written and post my findings and actions of it by the end of March. Who knows, perhaps more of the book will be delivered by that time and I'll be able to offer some more details when that happens.

This is what the book says it will ultimately include:

Table of Contents

• Foreword
• Introduction
• Why is testability important
• What does hard to test feel like
• What does testable feel like
• What leads to testability being neglected
• What is covered in this book
• How to use this book
• Feedback and suggestions
• 1. Use a testability inception deck to visualize current team and system state and create an environment for improvement
• 2. Adopt testability mapping to expose common smells of hard to test architectures
• 2.1 Gathering data on poor architectural testability to detect systemic problems
• 2.2 Low testability architectures contribute to slow feedback and deficient decision making
• 2.3 Identify the symptoms of poor architectural testability
• 2.4 Exercise: Measure the impact of testing smells on your architectural testability
• 2.5 Understand how testable architecture can impact your team’s testing efforts
• 2.6 Summary
• 3. Use risk and incident data to remedy architectural design problems which inhibit feedback from testing
• 4. Adopt ephemeral development environments to diversify testing techniques early and create shorter feedback loops
• 5. Utilize events and metrics to model risks in production for continuous improvement of your test strategy
• 6. Adopt incident postmortems to maintain a testability focus as part of your team’s continuous improvement strategy
• Terminology
• References
• About the authors
• Notes

It seems like a good place to start and I for one like to know I'm helping to fund progress on books I'd like to see be written. Win-Win!!!