Quantitative Usability Study
Say goodbye to guessing games – make UX measurable with the help of the Crowd.
QUANTITATIVE USABILITY STUDY – YOUR ADVANTAGES
Get faster results compared to qualitative usability studies
Make usability quantifiable and measurable
Easy, reliable comparison with competitors and other industries
Suited to your needs: standard questions, customized questions, or a combination of both
Leverage usability testing to bring the quality of your applications to the next level!
What is a Quantitative Usability Study?
The Quantitative Usability Study (QUS) is a user testing tool which gathers insights and validates usability by using quantitative data from your target group. In this service, testers from your target group will be asked to use core features of your digital product and take part in a quantitative survey. QUS is typically used to get feedback on the usability and design of your digital product – for example in comparison to a previous release or a competitor’s product.
In contrast to qualitative studies, quantitative methods measure the UX of your site through numbers such as satisfaction ratings, task-completion time, or task-completion rate. Quantitative methods help you answer questions like “How many?” or “How much?”. Unlike qualitative data, quantitative data helps you connect the user interface design of your digital product to numeric data. This way, you can make decisions based on statistically significant facts and measurements, rather than on gut-feeling or hypotheses. This helps you develop digital products with a user-centered design.
Usability is defined as the extent to which a product can be used by specified users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specified context of use (ISO 9241- 11). This means you need to measure three things to make an educated statement about the UX design of your digital product: effectiveness, efficiency, and satisfaction.
- Effectiveness is commonly measured by success rate – the percentage of users who managed to complete the given task successfully.
- Efficiency can be tested by calculating the average time it takes to carry out a specific task.
- Satisfaction, another important aspect of your user research, can be calculated by asking users to rate their satisfaction on a scale from 1 to 7, for example.
Scientifically validated benchmarks we offer
|NPS (Net Promoter Score)||Popular at the management level to measure the recommendation rate of the product|
|SEQ (Single Ease Question)||Simple but effective question to get feedback on difficulty at the use case level|
|UEQ (User Experience Questionnaire)||Semantic differential that measures six dimensions of the user experience (26 questions) - available in 22 languages; short version with ten questions available (UEQ-S)|
|AttrakDiff||Semantic differential that measures four dimensions of the user experience (28 questions); short version with eight questions available (AttrakDiff mini)|
|SUS (System Usability Scale)||Most widely used usability questionnaire|
| SUPR-Q (Standardised User Experience |
Percentile Rank Questionnaire)
|Questionnaire that measures the user experience using four dimensions and nine questions (currently only available in English)|
In addition, common metrics such as effectiveness (percentage of users able to successfully complete the use case) and efficiency (time taken for a user to successfully complete the use cases) can be measured.
If you plan to use other benchmarks or use others internally, you are welcome to make a request.
The Challenge: proof your UX and UI changes with numbers
You have the impression that the UI and user experience design of your digital product aren’t as good as they could be, but you need to prove that feeling with numbers. In particular, stakeholders or upper management usually use numbers as a base for their decisions, rather than assumptions. The re-design of a website, app, or any digital product can be quite expensive, so you need good reasons for starting that project and the proof for its return on investment.
Another challenge might be comparing differing designs, like an old and a new version of your website, or your product with the product of a competitor.
The Solution: make well-informed decisions based on research
Quantitative studies are the only studies in usability testing that help you put a number behind your UX design process, and therefore are the best instrument to calculate the return on investment of a new UI design, for example.
With a website usability study you can make well-informed decisions based on UX research. QUS can give you the much needed proof to convince your manager that a design relaunch is necessary, even if it’s expensive.
With a Quantitative Usability Study you gather insights and are able to validate usability by using quantitative data from your target group. In this service, testers from your target group will be asked to use core features of your digital product and take part in a quantitative survey.
This survey can be based on user performance on a given task like success rate, number of errors or task-completion time, but can also give you insights into the testers’ perception of the usability of your digital product with the help of satisfaction ratings. With Crowdtesting, you can select the exact members of your target group from our Crowd of over 400,000 testers, categorized by over 65 demographic criteria. This helps you get real customer feedback from real users, which paves the way to actual user-centered design (UCD) which takes all user requirements into account.
After the test, our project managers gather your customer feedback, analyze it, and give advice on how to optimize UI design and UX design of your mobile app, website, or digital product. That way we can secure user feedback that helps you develop an excellent customer experience and a customer-centric user interface.
Questions on a product’s user-friendliness are best asked to end-users themselves. Crowdtesting with Testbirds provides you with an unbiased testing group that matches your target group and thoroughly checks the usability and/or user experience of your software. Be it websites, apps, wearables, games, or smart devices: our testers determine how usable your software is in real-world conditions, and provide detailed feedback that helps you enhance quality. In addition, testing the entire user experience is possible and comes with multiple advantages. The test investigates all aspects that contribute to a positive or negative opinion of your company, from user’s expectations along the entire customer journey.
How it looks in practice
50 testers from your target group perform five core tasks on your website. After they finish these tasks, they rate their likelihood to recommend your website with a Net Promoter Score (NPS) – “How likely is it that you would recommend this website to a friend or colleague?” on a scale from 0 (not at all likely) to 10 (extremely likely). All testers responding to this question with a 9 or 10 are so-called “promoters”, testers responding with a 7 or 8 are “passives”, and testers responding between 0 and 6 are “detractors”.
To calculate the NPS, you simply subtract the percentage of detractors from the percentage of promoters, resulting in a score ranging from -100 to 100. There are benchmarks available, for example for consumer (B2C) and business software (B2B), to compare your results with. If your consumer product has a NPS of 5, you can put this number into meaningful context since consumer products have an average NPS of 23. This data indicates that your product is below the average ratio of likelihood to recommend, and should be improved in order to be able to compete on the market.
Solution Consultant QA & UX
Do you have questions about Quantitative Usability Study? Would you like to have more information on how we can help you make your digital products even better? Just send us a message, one of our Solution Consultants will be happy to get in touch with you.
Innovative App & Web Testing
Watch our masterclass video series on QA & UX Testing - as seen on The Telegraph & Business Reporter. Discover with our CEO & host Philipp Benkler how to optimize your digital products in the best way possible.