An alternative to Pairwise test design technique
Prehistory
Several years ago I made free to use online service for Pairwise test cases generation with allpairspy under the hood:
An article:
Note: This app doesn’t work today because I shutdown the server that hosted it because I had neither money nor will to pay for it.
And although you can find other pairwise tools here, I’d like to share with you an alternative for pairwise test design technique.
An alternative for pairwise testing
An alternative to pairwise testing — is questioning: why do we have so many options and combinations in the app to test? Is this really necessary? Show me the data that proves that this is necessary?
I am not joking! You don’t need to verify everything with everything because users don’t use everything. And to get what they really use you need analytics.
Alternative to pairwise testing is questioning: why do we have so many options and combinations to test?
And this is really a Product mindset question:
- Do they use all the options? (no, they don’t)
- Who of them uses and why do we care of them? (we don’t care of them equally, we care the most about already paying customers, so concentrate on their usage first)
Imagine if this is true (and I am convinced that it is true) and 50–80% of your app features are hardly ever being used:
What would you do?
Would you cover those 50–80% of features and options with testing all the combinations of parameters, writing and supporting manual and/or auto-tests, spending tons of resources to test this
OR
Will you go to your Managers and ask them: why do we keep all this shit that no one (but us) cares about in the app poisoning everything around?
If you apply feature usage analytics to your app and get usage stats like this:
then you can find those unused features, elements and options. With such Product UX Analytics you can concentrate your efforts on users and users groups you really care about, not on the “average user”.
And after that, when you remove those 50–80% of unused features, code and tests, then either you don’t need that pairwise anymore or you have x10 less combinations and you can test even without pairwise.
This is an alternative. And this is much harder than pairwise. But this is much deep and meaningful and Quality thing. I wanted you to know about this alternative when practicing meaningful and responsible Quality Assurance work.
If you like an article, please don’t hesitate to clap several times and to share it with colleagues. I believe we need more adequacy in this world.
