Exploratory Testing – An excellent complement to traditional software test approaches
Srikanth Renganathan (VP - Techpearl)
Introduction
Exploratory testing is an approach to software testing defined as simultaneous learning, test design and test execution. The term “Exploratory testing” was first coined and popularized by Cem Kaner in the early 1980s. Kaner is a professor of computer science and a leading authority on software testing.
My initiation to exploratory testing dates back more than a decade when I experimented with it to improve test coverage and ensure test sufficiency. For the telecom projects I handled, test sufficiency was a huge concern as the products developed were used by millions of people. At that time the literature around exploratory tests was very less and James Bach (A renowned Software Test expert and a pioneer in the field of exploratory Tests) was beginning to work in this area.
Exploratory testing is a type of software testing that involves actively exploring the application or system under test, without following a predetermined test plan or cases. The aim of exploratory testing is to uncover new and important defects in the application or system under test that might not have been discovered through traditional testing methods.
Exploratory Test Process
Exploratory tests were originally seen as an ad hoc test where you do random things and find issues. It was even equated with the term monkey testing. In reality, exploratory tests are much more subtle and profound. The tester here, ideally a person with good knowledge in the product domain is entrusted with this work. In our case, we used experts in the telecom domain to perform exploratory tests for telecom products in the Intelligent Network area. The process involves learning the system and testing the system simultaneously. Being the expert in the domain, he/she starts exploring the system most likely looking at a specific area more deeply based on the high risk areas identified by the product owner. Say for example the high risk area is security, he starts exploring the various ways in which the system could be compromised and tests various user scenarios. Once he identifies issues or vulnerabilities of the system he may go for more rigorous tests which can potentially break the reliability of the system.
From the process perspective, it has mainly three aspects involved.
a) Asking an expert to focus on specific areas of the system under test. It can be based on the risk, past issues in the system, customer usage scenarios etc
b) The exploratory tests are time boxed into sessions of 1 to 2 hrs each as very long hours bring saturation for a creative and innovative work like exploratory test
c) The tester is preferred to be domain expert to make sure the learning is fast and is sure when he finds an issue or defect
Gamifying Exploratory Test Sessions
We can gamify exploratory test sessions. In Techpearl, we have “Bug day” once a month and announce rewards to encourage participation. We identify the specific project for the particular month which will go through exploratory tests on the bug day. We identify the most high risk module or sub-system or feature set in that project and allow all employees who registered for this bug day event to have a go at it. Typically people from the same project, company level architects, people who have worked in the same project a while back etc participate in the event.
The bug day is divided into roughly 3 sessions of 2 hours each to explore and find issues in the system. It is very much like a hackathon in spirit. The participants have access to all relevant documents. The defects identified during bug day can be analyzed to see if they could be caught by traditional test methods. Bug day or Bug bashes are one of the creative ways to put exploratory tests in action. People have used mind-maps to implement exploratory tests and achieved successful results. It is up to us to design the right framework around exploratory tests suitable for our team or organization.
Summary
In my own experience, I feel exploratory tests are an excellent complementary test approach in addition to the traditional model of systematic tests based on test design and cases. Instead of the traditional static structure of test design, in this case the test case design is dynamic and happens on the run while exploring. Some companies even have titles for testers as exploratory testers. The main advantage of exploratory testing is that less preparation is required, and important bugs are found very fast. In summary, a software development organization irrespective of their size should consider exploratory tests as a key part of their overall test strategy and leverage its value.
