By Tam Bui
James Bach is one of the most famous testers in the world. He used to work for Dale Dishroon, Apple Computer, Borland, STLabs, SmartPatents, Reliable software technologies and now the CEO and Principle Consultant for Buccaneer-Scholar Incorporated and Satisfice, Inc.. James has deep understanding of software testing and knows how to execute software testing activities in an efficient way dependent on context. He is really an outstanding software testing expert in the software testing world.
When testers are assigned to test a new product, they need to learn about the product. They can learn it through user requirements, software requirement specifications, design documents, user guides, installation guides, standards, regulations, etc. However, learning from such documentations is not enough. There are actually some implicit documentations out there. In worse situations, products do not have any documentations, and testers need to explore and learn the product themselves. According to James Bach, exploring the product is a learning journey, and he suggests some exploring techniques which are categorized as below:
- Systematic Product Exploration:
- Levy Flight Heuristic
- Branching and Backtracking Heuristic
- Galumphing Heuristic
- Diminishing Returns Heuristic
- Fresh-Eyes-Find-Failure Heuristic
- Modeling from Observation
- Pattern Recognition from Experience
- Conjecture and refutation Heuristics connects to experimentation
- Alternation/Musing and Notetaking vs product interaction
- Specification writing.
- Product Element Labeling.
- Experiment Design
- One-factor-At-A-Time Heuristic
- Occam’s Razor Heuristic
- Patterned Data and Simplified Data Heuristic versus Naturalistic data
- Consistency heuristics
- Tool-Supported Testing Heuristic
I found many of these techniques useful. They can be used as the methodologies to explore the product. Eventually, using these techniques proficiently, testers can get a lot of information about the product. Details of the techniques will be described in other posts.
According to James Bach, testers can explore and learn the product by experience and inference. They need to think critically about what the product does. They use their experience to analyze, develop and infer deeper insight into the product behaviors. In order to gain good experience and inference, testers need to practice and improve their general and domain-specific knowledge. They need to learn, learn and learn as much as possible. They need to learn technologies and improve their technical skills. They need to improve their know-how. In addition, testers need to have regular conferences with stakeholders. In such conferences, testers can obtain more useful information about the product. This is why testers need to improve their communication skills to interact with different stakeholders. The last but not least is that testers need to have references to shared documentations/artifacts to get the big picture of the product.
That is not all. He also differentiates between tacit and explicit knowledge. Explicit knowledge can be represented completely in forms of a string of bits: words, pictures, and even actions. Tacit knowledge is not manifested in a form that can be equated to a string of bits: it is unspoken, unwritten, or unpictured.
According to Wikipedia, tacit knowledge is the kind of knowledge that is difficult to transfer to another person by the means of writing or verbalizing. For example, that London is in the United Kingdom is a piece of explicit knowledge that can be written down, transmitted, and understood by a recipient. However, the ability to speak a language, knead dough, play a musical instrument, or design and use complex equipment requires all sorts of knowledge that are not always known explicitly, even by expert practitioners, and which is difficult or impossible to explicitly transfer to other users. That is why testers need to socialize and work with customers, observe their daily tasks to understand and gain tacit knowledge which is very useful to understand the product’s descriptions and behaviors.
In the first days of exploring and learning the product, testers may get confused. A few days later, their minds will be lighted and they understand the big picture of the product. They will feel relaxed and enjoyed when the big picture of the product appears in their mind. On the way of exploring and learning, they may have concerns and questions about the product’s descriptions and behaviors.
When testers comprehend the product, they are suggested to write test cases. A test case is a set of ideas, instructions, or data for testing some part of a product in some way. Alternatively, a test case consists of specific detailed inputs and action steps taken during the execution with predetermined expected results. However, James Bach said test cases play no important role in Rapid Software Testing Methodology. Test cases are not testing.
James defines testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc. He said that testing cannot be encoded, meaning we cannot make a program to automate all testing activities. Similarly, it is just like a programmer who cannot encode all of his work or cannot write a program to automate all coding tasks. James also defines checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product. In this context, testing encompasses checking. Testing can be encoded partly and we call that encoded part as checking.
James said that a test means a plan for a test event or to create a test is to perform a test. A test cannot be an artifact which may be produced before, during, or after the act of testing. This is because the artifact does not think or learn in the full human sense of that word, that’s why, and thinking is central to the test process. That is why James Bach believes that test cases are not testing but testing is a live performance. The performance of evaluation of products by learning and exploring them through experimentation. So we should change the term test cases to check cases. What do you think?
In July 2016 conference, James Bach will present the topic ‘Testing Unexplained: The importance of Tacit Knowledge in testing and How to Develop it’. This talk will bring testers the world of tacit knowledge in software testing and help testers widen their mind to comprehend this special type of knowledge. Testers will know how to develop it and convert it to explicit knowledge then everybody can understand the product descriptions and behaviors precisely.