Showing posts with label testability. Show all posts
Showing posts with label testability. Show all posts

Thursday, April 20, 2023

Accelerating Quality with Conscious Deliveries: an #InflectraCON2013 Live Blog


So this is fun. Lalit and I have known each other for years. We have attended conferences together. I've been interviewed by Lalit for Tea Time With Testers. We've worked together within the Association for Software Testing on a variety of initiatives. Having said all that, I think this is the first time I've actually seen/heard Lalit speak :).

Lalitkumar Bhamare Avatar

Lalitkumar Bhamare

Tech Arch Manager, Accenture Song

It's interesting to realize that with all of the technological advances we have had over the past thirty years (probably longer but I've only been in the game for three decades), we still have to pick from the speed, cost, and quality triangle (you know it, "Fast, Cheap, Good. Pick two!"). It seems that if any shift is going to happen, it typically happens at the "Good" part, meaning that if any pressure comes into the situation, the quality side is the side that ends up bending. Granted, often that means we get "good enough" and for many people, that is sufficient. 

The irony is that we don't have to settle for good enough but it will require that upfront planning and resources be allocated to make sure that quality is reinforced. This comes down to requiring people to be motivated to provide not just good testing but a mindset of the importance of testing beyond the busywork of automation and declaring that testing has been performed. 

We had a little discussion about apps we actually like using. Recently due to life circumstances, I've become more familiar with healthcare apps. My company's insurance provider has done big on telemedicine recently and to that extent, it seems they have either found out I'm a tester or I just tick off a lot of boxes for them because they have thrown just about every app and tool my direction to manage healthcare and treatment options from a digital perspective. Some of these apps have been really helpful and some of them have been... less than desirable, to say the least. To be clear, these apps are not developed by the insurance provider but they are either partnerships or investments that my insurance provider has made and encourages use. It's interesting to compare them and see what makes them "quality" products vs. not-so-good quality. Also, some apps have great quality in some areas while being less good in others. A perfect example of this is an app I'm involved with that focuses on weight management specifically for people who are at risk for Type 2 Diabetes (which family history points to me being, so I'm part of their initiative for that reason). In areas where real human interaction takes place, it's great. However, they have recently made decisions to limit website updates and almost exclusively manage via their mobile app. In one way, this makes sense, as we are more likely to be within arms reach of our phones at meal breaks as compared to our computers. Still, I type way faster on a computer than I do on my phone, so invariably, their change has resulted in my updates being delayed, sometimes by days, because it's less convenient for me to enter the details. I'm curious if anyone on their team even brought up this possibility.

Lalit emphasized three "P" areas of quality consideration. You have a Project aspect, a People aspect, and a Product aspect. He additionally emphasizes the 4 "E"s of quality. Enable, Engage, Execute, and Evaluate. The 4Es apply to each of the 3Ps. Granted, each of these elements has a context based on where it is applied and there are biases that come into play, it uses the story of the "Parable of the Elephant" where our limitations often constrain our vision and view of an aspect of something (I had a good laugh realizing that Lalit chose Dieter F. Uchdorf's telling of this story. It took me a minute but I kept thinking "wait, I know that voice" (LOL!) ).

Over time, we need to be open to learning new things and incorporating more knowledge of the potential for requirements, and translating that to concrete actions that can be used. As testers, we may be doing a lot of work but we may not actually be aligned with actual business issues or challenges. As I love to borrow from Steve Covey, how infuriating is it to know we've climbed a significantly tall ladder only to realize we've placed it against the wrong wall?

We know that we cannot engineer quality, at least not in a literal sense. What we can do is preserve as much of the product's intended integrity as possible and take steps to make sure that we are learning and focusing on areas to make sure that we are creating the best product we can. To that end, having quality experience sessions can help to inform how a product is being used and what can be done going forward. Ideally, these considerations are made early on in the life of the product or as it is being developed. For that to be effective, it requires people with a focus on quality to ask questions and experiment with requirements early on. The later this happens, the less likely they will be of value in design but it might be very demoralizing to realize the "right ladder, wrong wall" problem is happening after we've climbed quite a bit.

To wrap this up, if you need to have a simple thing to consider and practice, "test early, test small, and test continuously" is a pretty good approach, and apply it to all of the areas you interact with. if you find it valuable, share the approach and help it expand through the organization.


Wednesday, November 10, 2021

Is It Testable? A New Episode of The Testing Show is Now Available

 

Image placard for The Testing Show: Episode 107: Is This Testable?

It's that time again. Another episode of The Testing Show is now available. Last week we posted "Is This Testable?" This was a fun interview that Matt and I had with Gil Zilberfeld and as an added bonus, I was able to add some input as a co-quest this go around (as Testability is one of my talking points :).


I'm trying an experiment, so I'm hoping this works. I'm embedding the show in these posts and seeing if they appear when I post them. If not, here's the link to the episode page.

As a follow-up, it was interesting to hear where Gil and I shared ideas and where we differed. Well, saying we "differ" isn't correct. More accurately it was interesting to see what we tended to prioritize of where we focused our respective attentions :).

Anyway, the show can tell you lots more than I can/should, so go have a listen, or a read if a transcript is more up your alley ;).

Thursday, November 4, 2021

“Make it public!” And other things that annoy developers about testability



This is a fun session as I have already had about an hour-long conversation with Gil about this very topic for the next The Testing Show (hopefully to be out shortly :) ). Testability is also a bit of a pet topic of mine so it's really cool to hear Gil talking about this directly.

Testability is often of those areas that can be hard to implement late in the game but is much easier to deal with earlier in the process. It's not a common topic in development circles or at least that may seem to be the case. Sure, developers are encouraged (and in many cases required) to work on unit tests. However, unit testing does not translate to overall testability. Testability is putting hooks or developing a product to be able to actually be easier to share information in ways that can help a tester make a decision as to a feature's genuine fitness for use.

Another challenge we all face is the fact that developers are not interchangeable. What one developer does that may be beneficial may be totally outside the wheelhouse of another developer. Additionally, one developer may do very well in delivering their pieces of code but struggle to integrate with other components. Testers can often help fill in the blanks but often we run into places where a system that has been developed is frustratingly opaque. Often it takes for granted we will be looking at an application with a UI. Many of those interactions, to actually look at the workflows in question, may require the user to set up their tests to interact with the UI multiple times just to get to a place that's useful. 

An area that Gil and I agree on is the fact that testers if they want to advocate for testability, need to at least have a working knowledge of and the ability to scan and read code. Not necessarily writing code at a developer level but having enough proficiency to read code and get the gist of what's happening can be very helpful. Let's take a classic example, look at the error handling code and see what error messages have been coded in. Can you surface those messages? What would it take to do so? In this circumstance, I'd probably sit down and see if I could surface each of those messages at least once. If I can't do so, I would then get back with the developer and say "what will it take to surface this?" In some cases, it's Oh, right, you need to configure this or do it this way" but sometimes we've been able to prove we can't really get there. These are examples where, if there were API hooks that we could interact with, we could more easily determine if they could be properly surfaced or if they were perhaps not necessary.

One of the other areas Gil points to as frustrating both from a Development and testing standpoint is code duplication. There's of course refactoring to remove used code in places but that duplicate code doesn't just end up being a drag on development. IT can also stymie testing efforts because we may think we have a fix in one place only to discover another place is still broken because the code that got fixed on one place didn't get fixed here. This isn't as simple as adding a hook or putting an API call in but it's definitely a testability issue that needs to be discussed.

The key point Gil is making is the fact that we should make our interactions public. Specifically, we should have an API that we can use for testing purposes and also for active production uses for those who want to do it. "But what if people call those APIs after we ship?!" Well, what about it? If exposing something via an API literally breaks your application, how robust is your original design? More to the point, does the risk of exposing elements and methods outweigh the benefits of easier testability and potentially more usability for the customers? I can't answer that for everyone but from personal experience, making items available in the API and adding testability hooks has not come back to bite us later. In fact, it has actually benefitted us overall by giving our users other options they can use to complete their workflows, in some cases not even interacting with the application at all. 

The key takeaway here, IMO, is that we as testers can absolutely influence this conversation and the earlier we get involved in this conversation (And encourage mocking, API calls, dependency injections, etc.) can help us test better and more comprehensively. The critical element, though is that we need to be involved in this process early/ The sooner we can get into a story being developed, the more likely we can have a positive effect on testability. The longer we wait, the more resistant the process of testability will become, not just from the developer's perspective but from the literal code perespective ;).


Monday, October 14, 2019

Is This Testable? - my talk at #PNSQC2019

All right, one down! My talk, I think, went well.

My huge thanks to Jennie Bramble (@jenniedoesthings) for live-tweeting basically my entire talk. Thus, I'm going to capture for posterity her stream of consciousness and again offer my enthusiastic gratitude :).
-----


an hour ago27 tweets, 23 min read
 
It's time for Is This Testable? A Personal Journey to Learn How to Ask Better Questions From My Applications and Engineering Team with @mkltesthead at @PNSQC #PNSQC2019 
Also, I'm taking selfies with his phone. :D @mkltesthead #PNSQC2019 

mentions I like playing dress up in many forms. @mkltesthead #PNSQC2019 
mentions It's a voyage of discovery! @mkltesthead #PNSQC2019 
mentions The @ministryoftest 30 Days of Testability just got a shout out by @mkltesthead #PNSQC2019 
mentions Testability involves a few things: communication, involvement, and a willingness to look at a product objectively. @mkltesthead #PNSQC2019 
mentions A hypothesis is testable if there is some real possibility of deciding wether it is true or false based on real experience. @mkltesthead #PNSQC2019 
mentions What is light? What is dark? We don't really know, so we can't really test it. But if we use actual numbers for light and dark, then we can test if they are too close. @mkltesthead #PNSQC2019 
mentions "Generally speaking, math is pretty consistent. We'll leave string theory out of this." @mkltesthead #PNSQC2019 
mentions You can't automate a subjective feeling. You can automate objective things. @mkltesthead #PNSQC2019 
mentions Get familiar with your log files! Welcome to your first flash of the blindingly obvious. @mkltesthead #PNSQC2019 
mentions We can all agree this is generally how cars work. Tesla owners, don't get at me. @mkltesthead #PNSQC2019 
mentions We have to look at the things that plug into our application--each component has log files that I need to keep track of. @mkltesthead #PNSQC2019 
mentions The takeaway I want to encourage is to champion unified log files wherever you can. @mkltesthead #PNSQC2019 
mentions How many of you have a good relationship with your sales engineers or customer support engineers? Only a few? Okay, homework: go home, find one, buy them a drink, and learn about their pain points. @mkltesthead #PNSQC2019 
mentions If you can collaborate with someone like a sales engineer for CSR, they can help you have the best test environment available.@mkltesthead #PNSQC2019 
mentions Being able to plug in feature analytics will let you know how people are adopting your features. What features are they actually using? @mkltesthead #PNSQC2019 
mentions We can guess, but real data is much better. @mkltesthead #PNSQC2019 
mentions Automat-able does not necessarily mean testable. @mkltesthead #PNSQC2019 

mentions Second piece of homework: everyone go look up @eviltester and follow him. @mkltesthead #PNSQC2019 
mentions "How am I going to test this?" If you don't ask this for every feature, you will regret it. @mkltesthead #PNSQC2019 
mentions "What are situations I can throw at this problem that go beyond the status quo?" @mkltesthead #PNSQC2019 
mentions I'm a reenactor and a cosplay nerd--I love persona testing! @mkltesthead #PNSQC2019 
mentions Conduct regular show and tell--show people what you've been working on! @mkltesthead #PNSQC2019 
mentions Pair with your developers. Work on tests together! @mkltesthead #PNSQC2019 
mentions Identify your dependencies. Your application is not just your application--it is only as testable as he most remote dependency you have to interact with. @mkltesthead #PNSQC2019 
mentions Heat seeking: check your source code and find where you're making frequent changes. @mkltesthead #PNSQC2019  

Sunday, October 13, 2019

En Route to Portland - a #PNSQC2019 "Live Blog"


It's been a while since I've done this.

As I sit in San Francisco International Airport, I am laughing at the change this year has brought about. One o the more interesting ones is just how many times I actually come and go from this airport every day, albeit for different reasons than most. My daughter started working or SFO during the summer. Should you fly on Scandinavian Air or  Air New Zealand, you might get to interact with her :). However, it's because I'm flying out today that I'm seating and gearing up the little blog engine that hopes to.

Yes, it's time for the Pacific Northwest Software Quality Conference, one of my officially most attended and participated in conferences. I will be giving a talk on Monday morning called "Is This Testable? A Personal Journey to Learn How to Ask Better Questions From My Applications and Engineering Team". It's all about my adventures in Testability Land. What, you expected an Accessibility talk ;)? I can't say I blame you as that's been my go-to advocacy for the past five years but this year I decided to try something a little different. This time I'm going to share the details of my experiences with looking at Testability and what I learned from that process. If you are attending PNSQC and want to check it out, here's the details and a link to my paper:


Also, tomorrow at Lunch, Bill Opsal and I will be playing a little game with those who want to attend called "The Release Day Crime Scene":

Join Michael Larsen and Bill Opsal for a game of sleuthing. It is release day and a crime against code and customers has been committed. Can we help identify who done it? Take part in this interactive panel and discuss ways to identify and examine clues, build a case with evidence and make the streets safe for functions and methods.

All in the name of truth, justice, and the releasable way.

So, yeah, if you are at the conference, I look forward to seeing and talking with you. With that, it looks like it's time to check-in and get ready to board. Next stop, Portland :).

Sunday, March 31, 2019

Book Review: Team Guide to Software Testability: a #30DaysOfTesting Testability #T9y Entry

Woohoo!!! Day 30 completed on March 31! Truth be told, this was a bit much and I have no one to blame but myself for how this turned out. Still, I feel like I learned a lot and covered a lot of ground. Some of it felt familiar but there was quite a bit of new information and perspectives I had a chance to look at and think about how to implement with my team. It's time to bring the formal "30 Days of Testability" to a close and to do that, I'm going to review a book that, as I came to the Ask Me Anything post, made me realize that the sudden availability of this book wasn't an accident, as one of the authors of the challenge, Ash Winter, was also one of the writers of this book. How convenient :)!

What book did you choose on Day 3 and what did you learn from it?


The book I chose was "Team Guide to Software Testability" by Ash Winter and Rob Meaney.

This version was published with Leanpub on 2019-03-08. Again, mighty convenient :)! While the book is listed as being 30% completed, there is plenty to consider and chew on with regards to testability for apps and navigating how to approach testability for you, your team and your applications. To borrow a line from the Introduction:

"We want to show that testability is not only about testers and, by extension, not solely about testing. It is almost indistinguishable how testable your product is, from how operable and maintainable
your product is. And that’s what matters to your (organisation’s) bottom line, how
well your product fits your customer’s needs."

Testability goes hand in hand with predictability. When an application responds in a predictable manner, we are able to interact with better certainty that we will see the results we expect. That, in turn, helps inform the testability or lack thereof of our applications.

Testability Mapping allows users and teams to get a better feel for the areas of their application that are working well and those that still need to be worked on. When we have a low understanding of the underlying testability architecture, we often struggle with anything approaching effective testing. To help address that, it's important to take stock of the testability of our applications and see how far afield that might take us (applications are rarely monolithic today, they have dependencies and other components that may or may not be obvious.

Our testing environments should be set up and configured in a way that allows us the maximum understanding of its underpinnings. Doing so lets us get started with testing and receiving meaningful feedback from the application. Paying attention to these environments and periodically addressing the testability helps make sure that complacency doesn't come into play. Environments are not static, they grow and develop and the technologies that are good for one period of time may be inadequate later.



Bottom Line:

Even at 30% complete, there is a lot of good information in this book. Is it worth purchasing as it currently is, with the idea that more will arrive over time? I say "yes". If the idea of helping your team develop a testability protocol sounds exciting and necessary, there is a lot to like in this book. Check it out!!!


Grinding Halt: a #30 DaysOfTesting Testability #t9y Entry

So close to the end! Just a couple more to go and I'll be able to call this challenge "surveyed". I won't really be able to call it "done" or fully "completed" until all of these aspects are put into place and we've moved the need in regard to their being examined. Still, this has been an active month, with a bit of an overly-ridiculously active final week (alas, nobody to blame for that but me). Nevertheless, for those who want to play along, check out the "30 Days of Testability" checklist and have some fun with me.

Do you know which components of your application respond the slowest? How could you find out?

This is a tricky question as there are different levels of what could be causing slowness. In the first instance, there are native components within our application itself, as in not those that we consume by other groups through microservices. Honestly, the ability to render full wiki pages as widgets with all of their formatting makes for some interesting interactions, some of them take time to pass along. As we can render some seriously complex HTML and CSS in these pages, to then have that displayed as a widget and then to have that displayed in a Responsive interface, it just takes time and in certain instances yes, it can be felt.

Other areas are a little harder to define but fortunately, we do have a way to determine how long it takes. Our application has a scheduler that runs in the background and every major interaction gets logged there. Want to know how long it takes to process ten thousand users and add them to the system? I can look that up and see (hint: it's not super quick ;) ).

The other areas that are challenging are where we consume another product's data via microservices to display that information. This isn't so much an issue of fast vs. slow as it is an issue of latency and availability. Sometimes there are things beyond our control that makes certain interactions feel "laggy". On the plus side, we have similar tools we can use to monitor those interactions and see if there are areas we can improve the system and network performance.

Watching the Detectives: a #30DaysOfTesting Testability #T9y Entry

I realize I probably should feel a little bit shameless right now at the silly puns I use for many of these titles. I should... but I don't ;). Besides, we are almost done and next week you can watch me shift gears and Live Blog about STP-Con, so just bear with me for a couple more "30 Days of Testability" posts :).


Pair with an internal user or customer support person and explore your application together. Share your findings on The Club.


This is actually a pretty recent experience for me in that through our LTG merger I was acquainted with a new manager, a new overall test director, a new VP of Engineering, and a CTO all at the same time. By interacting with each of these people, I've had the chance to show what Socialtext is, what it isn't, and what we need to do so that we can get all of our moving parts to work in context together. Each interaction helped me see both the areas that people understood about our product and areas where there were some gaps in understanding or context.

Because of the nature of how we configure our product suite (this is going outside of Socialtext now) we had several people discussing our product (the broader PeopleFluent/LTG product offerings in this case) and when it clicks with people exactly what our product does and how it does it, there's this smile and a reckoning of what goes where and why. Don't get me wrong, sometimes there are other emotional reactions other than smiles but those are thankfully the more common ones. Additionally, it's also neat to see what other arms of our broader product do and how they interact with and integrate with our platform. Consider this a strong "two thumbs up" to demoing and looking t as much of your product with as many people from as many business interests as you can. My guess is you will learn a lot from those interactions. I certainly have and will most likely continue to :).

Stop Me If You Think That You've Heard This One Before: a #30DaysOfTesting Testability #T9y Entry

Hah! I 've finally found a way to work that title into a TESTHEAD post (LOL!). Seriously, though, I'm hoping that after this flurry of posts, you all still like me only slightly less than you used to... OK, enough of that, let's get back to the "30 Days of Testability", shall we?

Use source control history to find out which parts of your system change most often. Compare with your regression test coverage.

I already know the answer to this since it's been a large process. Our most changed code is literally our front end. The reason is straightforward. We are redesigning it so that it will work as a Responsive UI. that means everything is getting tweaked around the front end. Our regression testing, therefore, is in the spin cycle. It's getting majorly overhauled. Our legacy interface, on the other hand, is doing well and will still be there for those who choose to use it, so that is adding an exciting challenge as well.

the biggest challenge I am personally facing is that the tests we have for our legacy interface are solid and they work well, but they are almost totally irrelevant when it comes to our Responsive interface. the IDs are different, the rendering code is different, the libraries that are used are different. the workflows are similar and in many ways close to the same but don't quite lend themselves to a simple port with new IDs. Thus I'm looking at the changes we are making and figuring out how we can best automate where it makes sense to. Needless to say, it's never dull.