While Linux fans attribute the quote used in the title to Linus Torvalds, Windows supporters argue that it was Bill Gates who said it, while Apple admirers swear it belongs to Steve Jobs. Fortunately, for us, the author is of little relevance. The differentiation, on the other hand, undoubtedly is. Every person makes mistakes and developers are no exception. Sometimes, though, what we accidentally perceive as a mistake is in fact completely intentional. In its past, Testbirds has had numerous encounters with this misunderstanding. Therefore we would like to explain, step by step: Is it a bug or is it a feature?

There is a rumor that in 1947 a real life bug, meaning insect, was trapped in a relay in Harvard’s Mark II computer, thereby coining the word “bug” in the IT industry. In that context, a bug is defined as “an error, flaw, failure, or fault in the computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways.” Practically speaking, though, what makes a bug a bug? Trust me; we are veterans when it comes to test reports. All kinds of test reports. Reports that provide too little information, reports filled with wrong information, reports which state problems that only occur because of wrong software usage. But also, wonderful, clear, helpful and informative reports!

The rule of thumb here is: “If your bug is not reproducible, it will never get fixed.” Therefore, it is good practice if after a bug is found, you try to make it appear again. We do like problems that are reproducible, and happy Testbirds can analyze reports faster and more effectively.

Malfunctions, app crashes, spelling mistakes and display errors

At Testbirds, we use several categories of bugs: there are for instance malfunctions, app crashes, spelling or grammar mistakes and display errors. Malfunctions and app crashes are usually more critical: they vary from medium to extremely important. To find out more about those, take a look at our software fails series …they are bad. Spelling/grammar mistakes and display errors are generally classified as lower in severity. If pictures and buttons are sticking, even though the page has been scrolled down, or there is overlaid content, this is called a display error.

Sounds simple, right? However, these guidelines are not strict and that’s the tricky part: it is always important to use common sense instead of adhere to stringent guidelines. If a company is mistyping its own name on the main page, it should not be considered a less important mistake! This is why in addition to the ranking from low to critical, it is utterly important to use questions that help our Birdmasters rank a bug’s severity. Is there a workaround?

Does the bug affect main functions? Those and many more questions might make the Birdmasters give preference to bugs that are generally classified as low in severity.

What does this mean for testing?

However, let’s think of a few other examples: What if the text isn’t shaped nicely around the picture or if the text is unreadable because it is too small or in the wrong color? If you report a suggestion for change for the existing or new functions to make the application better, it’s called usability issues, which, strictly speaking, are not bugs. And what if you are supposed to test an app in flight mode and various functions no longer work? What about if the app automatically switches to landscape mode? If the design changes?

Those things might confuse the tester but are meant to be a “feature”. We call that the Testbirds dilemma: we want our testers to look at software from an unbiased perspective and we also don’t ask our clients for any software specifications. This means that borderline cases can happen, where it is hard to define whether a bug reported by a tester is actually a bug, a feature or a usability issue.

One possible way to handle this is to accept the bug report with a low severity and in the cases of a BugAbility™ test the problem should also be mentioned in the usability part of the testing process.

So the number one rule is: no machine in this world can replace human common sense as there will always be exceptions.

Franzi

Bugs made by Freepik from www.flaticon.com is licensed by CC BY 3.0