Wednesday, June 9, 2010

Wednesday Book Review: The Structure of Scientific Revolutions

 Through the years, I have come across a number of books that I have used and valued. These may be new books, or they may be older ones. Each Wednesday, I will review a book that I personally feel would be worthwhile to testers.

Wow, I’m really going back in time for this one (LOL!).

The Structures of Scientific Revolutions by Thomas S. Kuhn is recommended reading by both James Bach and Michael Bolton and mentioned in their Rapid Software Testing syllabus notes. It was here that I first heard of this book, and decided it would be an interesting read. Interesting? Yes. Easy? No!

This book was written almost 50 years ago, and there have been many “Scientific Revolutions” since, from plate tectonics to human genome sequences, and other advances that have added to the premise of Kuhn’s 1961 original edition. Kuhn’s work was to show that science and scientific discovery is often shown to neophytes and lay-persons as though it were an orderly march from one theory to another. In fact, scientific discovery and the “shifting of paradigms” (a term that Kuhn himself made famous in the succeeding decades) often goes through fits and starts, where one model of scientific inquiry works for awhile, and progress is made, but at some point, a crisis point is reached where the problems being faced can’t be solved the same ways with the same tools. To progress, a new way of thinking is required. The world didn’t change, just our perception of it, and that change in perception makes a huge difference. It’s also frequently fought tooth and nail by the very established practitioners of old ways, or at least until enough consistent proof of the concepts comes through that they themselves join with the new paradigm (or in some cases stubbornly don’t).

Why would I recommend this book to testers? Having had a chance to see James and Michael’s work and get to become more familiar with the approach towards “context based” testing, this is a book that really helps a person delve into the ideas of context and how context applies to all endeavors (science, philosophy, arts, business, software development, and yes, testing). To borrow from Steven Covey “We don’t see the world for what it is, we see the world for who we are”. In earlier times, humans could model their world where the Earth was the center of the universe and all things revolved around it. It’s not an accurate view, but much was able to be done with that model (sailors learned to navigate by the stars, super structures like the pyramids of Egypt and the Ziggurats of Mesopotamia were built, mathematical proofs still in use today were developed). In time, with the development of more sensitive tools, of finer measurements and a better understanding of principles, this view was called into question, and Copernicus finally had enough traction to make waves enough to cause the tide to turn in favor of a helio-centric solar system, of which the Earth is a part.

This is one of many “Scientific Revolutions” that Kuhn describes. Other examples include Galileo and the development of optics sophisticated enough to see details of distant planets and to discover asteroids, comets, and stars that were never observed before, making the universe a much larger place. The developments of Isaac Newton and the shaping of the modern science of physics is given a good shaking, and the developments are seen in their historical contexts (and not so unanimous at their time). Newton’s laws worked well, until they didn’t, and then another model was put forward to help fill in the gaps. That model was Einstein’s theory of relativity, which today is both lauded and criticized. Note that Newton’s laws did not suddenly become extinct; we managed to get to the moon and back using Newton’s theories.

Interestingly enough, up until 200 years ago, it was possible to be an active chemist and not believe in the existence of the atom. Kuhn goes to great lengths to describe the change in practice from a world where Phlogiston exists to a world where Oxygen exists. Note that the world didn’t change, but our perception of it did. The key element in Kuhn’s philosophy, as I see it, is that we tend to look at science as the discovery of “the truth” about our world, and that each generation shows the false traditions of the previous generations, and that they have discovered “the truth” this time around. Kuhn makes the case that “The Truth” changes as the paradigm for seeing the world changes. It’s also important to realize that a world view greatly colors what “truth” we see, and that often we will discard “truth” that doesn’t fit that world view. Thus it is important to try to see things in different ways whenever possible, and to take the opportunity to change the context that we view the problems we face. By changing the context, it is possible to see different solutions. Getting others to go along with those changed perspectives then becomes the challenge. Get enough to do it, and the world (or universe) can change.

One thing I must warn anyone who wants to read this book… if overly academically worded presentations bore you, this book will be a slog. It’s written in a very academic tone, and so often words used come across in a clumsy manner (hey, I’ll confess that I like things I read to be more conversational, and this definitely isn’t. For a 200 page book, it reads like a 600 page book for pace). The first few chapters may be challenging, but if you can get through them, the pace picks up considerably and the scientific issues and their times take center stage. Kuhn at times explains the background so that novices can appreciate what’s going on, but some areas take awhile to understand, and I found myself going to Wikipedia a few times so I could find out who Kuhn was talking about in certain sections (he is presenting this work to an academic body that is already intimately familiar with the material, so it’s not a surprise that certain areas do not get covered). Once you get past these areas, however, then the book becomes very interesting and many crisis points in the history of science are seen and how those crises are resolved is documented. There is a postscript hat was written in 1969 that clarifies a number of the thoughts and areas in the original version, but the premise is still the same; science, and scientists, do not see the world as it is, but as they are, and succeeding generations find more sophisticated ways and means to speak to the challenges they face, and they construct models that better help to explain and understand them.

The Structure of Scientific Revolutions is an opportunity for the reader to take a close look at what knowledge actually is, inside of the world of science, and ask themselves the fundamental questions of epistemology: What is knowledge, how is it acquired, what do people really know, and how do we know that we really know it? Is Structure of Scientific Revolutions a perfect example of this? No, but it definitely gives some great food for thought, and shows us that the neat and tidy progression that science is often portrayed as has often through history been anything but. For those looking for a good “shake up of the system” and a good primer in how to look at things from different perspectives, Structures is a pretty good starting point.

2 comments:

Justin Hunter said...

"Kuhn on Kuhn"

A couple comments:

1) On the theme of scientific stuff interesting to testers, I'd also highly recommend the a DVD set or two on "Connections" which Michael Bolton recommended to me as the best television programming he's ever seen. Each show involves a series of scientific advances and/or discoveries that are wonderfully woven together in a pleasing ADHD-like fashion. Following the story arc from one invention to the next is fun, brain-engaging stuff for 10 year olds and science PhD's alike. The shows are significantly more "low brow" than the Kuhn book you reviewed but significantly more "high brow" than, well just about anything that's shown on commercial television these days.

2) "Kuhn on Kuhn" explained... I co-wrote a paper with Rick Kuhn (a PhD at the National Institute of Standards and Technology / NIST) last year on pairwise and combinatorial testing methods that included a small, scientific study on software test design effectiveness. For analytically minded readers, the findings were really quite compelling.

We found that, on average, combinatorial test design techniques led to testers finding 2.4X as many defects per hour (and 13% more defects overall) when that test design approach was compared to what happened when testers executed test cases that they selected manually. I have since seen examples where there would be smaller efficiency and effectiveness improvements (and a recent example where combinatorial test design would be a poor strategy to use as a starting point), but still stand by those numbers (which were an average benefits from 10 real-world projects).

I recommend it as pragmatic, science-inspired reading for testers. (Hypothesis --> Experiment --> Data --> Conclusion --> Revision and Refinement of Hypothesis). I wish there were more experiment-driven, factual data-based articles out there on software testing.

The article is available at: https://www.hexawise.com/Combinatorial-Softwar-Testing-Case-Studies-IEEE-Computer-Kuhn-Kacker-Lei-Hunter.pdf

or, if you don't like direct pdf download links, through hexawise.com/casestudies

Michael Larsen said...

Hi Justin.

Thanks for the comment, and thanks for the additional information and links. I will *definitely* check it out.

As to "Connections", I dig James Burke and his approach to "history mapping". It's very fun and I thank you for jogging the memory regarding it... now I'm jonesing to see some episodes :).

--MKL