Select Page

Quantitative Usability Study

Say goodbye to guessing games – make UX measurable with the help of the Crowd.



Get faster results compared to qualitative usability studies


Make usability quantifiable and measurable


Easy, reliable comparison with competitors and other industries


Suited to your needs: standard questions, customized questions, or a combination of both


Leverage usability testing to bring the quality of your digital products to the next level!


What is a Quantitative Usability Study?

The Quantitative Usability Study (QUS) is a user testing tool which gathers insights and validates usability by using quantitative data from your target group. In this service, testers from your target group will be asked to use core features of your digital product and take part in a quantitative survey. QUS is typically used to get feedback on the usability and design of your digital product – for example in comparison to a previous release or a competitor’s product.

In contrast to qualitative studies, quantitative methods measure the UX of your site through numbers such as satisfaction ratings, task-completion time, or task-completion rate. Quantitative methods help you answer questions like “How many?” or “How much?”. Unlike qualitative data, quantitative data helps you connect the user interface design of your digital product to numeric data. This way, you can make decisions based on statistically significant facts and measurements, rather than on gut-feeling or hypotheses. This helps you develop digital products with a user-centered design.

Usability is defined as the extent to which a product can be used by specified users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specified context of use (ISO 9241- 11). This means you need to measure three things to make an educated statement about the UX design of your digital product: effectiveness, efficiency, and satisfaction.

  • Effectiveness is commonly measured by success rate – the percentage of users who managed to complete the given task successfully.
  • Efficiency can be tested by calculating the average time it takes to carry out a specific task.
  • Satisfaction, another important aspect of your user research, can be calculated by asking users to rate their satisfaction on a scale from 1 to 7, for example.

Scientifically validated benchmarks we offer

Type Content
blank   NPS (Net Promoter Score) NPS is one method in software testing to gauge the satisfaction of your users. Net Promoter Score is calculated on the question of how likely it is that a customer would recommend a company, product, or service to a friend.
blank   SEQ (Single Ease Question) As the name suggests, SEQ is based on one single question which asks the users how difficult it was to complete the given task. The scale ranges from 1 (very difficult) to 7 (very easy).
blank   UEQ (User Experience Questionnaire) UEQ measures the attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty of a digital product.
blank   AttrakDiff AttrakDiff helps you record the perceived pragmatic quality and hedonic quality of your digital product. It consists of 28 prompts of opposing adjectives (easy – hard, confusing – clear, ordinary – novel, and so on).
blank   SUS (System Usability Scale) SUS is a simple and technology-independent questionnaire that helps you assess the usability of a system. It consists of 10 questions on a Likert scale (statements that respondents agree with or disagree with on a given multi-level response scale).
blank   SUPR-Q (Standardised User Experience
      Percentile Rank Questionnaire)
SUPR-Q is used in user experience research to measure the UX of a website with a questionnaire of 8 questions.

In addition to these benchmarks, common metrics such as effectiveness (percentage of users able to successfully complete the use case) and efficiency (time taken for a user to successfully complete the use cases) can also be measured.

It’s very important to know that all quantitative usability methods need a reference point to produce valuable results. That means that you need a given KPI which is suitable as a comparative value. This could be a finding from a former webpage test or an industry standard – such as the average SEQ score which is around 5.5.


The Challenge: proof your UX and UI changes with numbers

You have the impression that the UI and user experience design of your digital product aren’t as good as they could be, but you need to prove that feeling with numbers. In particular, stakeholders or upper management usually use numbers as a base for their decisions, rather than assumptions. The re-design of a website, app, or any digital product can be quite expensive, so you need good reasons for starting that project and the proof for its return on investment.
Another challenge might be comparing differing designs, like an old and a new version of your website, or your product with the product of a competitor.

The Solution: make well-informed decisions based on research

Quantitative studies are the only studies in usability testing that help you put a number behind your UX design process, and therefore are the best instrument to calculate the return on investment of a new UI design, for example.
With a website usability study you can make well-informed decisions based on UX research. QUS can give you the much needed proof to convince your manager that a design relaunch is necessary, even if it’s expensive.
With a Quantitative Usability Study you gather insights and are able to validate usability by using quantitative data from your target group. In this service, testers from your target group will be asked to use core features of your digital product and take part in a quantitative survey.

This survey can be based on user performance on a given task like success rate, number of errors or task-completion time, but can also give you insights into the testers’ perception of the usability of your digital product with the help of satisfaction ratings. With Crowdtesting, you can select the exact members of your target group from our Crowd of over 500,000 testers, categorized by over 65 demographic criteria. This helps you get real customer feedback from real users, which paves the way to actual user-centered design (UCD) which takes all user requirements into account.

After the test, our project managers gather your customer feedback, analyze it, and give advice on how to optimize UI design and UX design of your mobile app, website, or digital product. That way we can secure user feedback that helps you develop an excellent customer experience and a customer-centric user interface.

Questions on a product’s user-friendliness are best asked to end-users themselves. Crowdtesting with Testbirds provides you with an unbiased testing group that matches your target group and thoroughly checks the usability and/or user experience of your software. Be it websites, apps, wearables, games, or smart devices: our testers determine how usable your software is in real-world conditions, and provide detailed feedback that helps you enhance quality. In addition, testing the entire user experience is possible and comes with multiple advantages. The test investigates all aspects that contribute to a positive or negative opinion of your company, from user’s expectations along the entire customer journey.


How it looks in practice

50 testers from your target group perform five core tasks on your website. After they finish these tasks, they rate their likelihood to recommend your website with a Net Promoter Score (NPS) – “How likely is it that you would recommend this website to a friend or colleague?” on a scale from 0 (not at all likely) to 10 (extremely likely). All testers responding to this question with a 9 or 10 are so-called “promoters”, testers responding with a 7 or 8 are “passives”, and testers responding between 0 and 6 are “detractors”.

To calculate the NPS, you simply subtract the percentage of detractors from the percentage of promoters, resulting in a score ranging from -100 to 100. There are benchmarks available, for example for consumer (B2C) and business software (B2B), to compare your results with. If your consumer product has a NPS of 5, you can put this number into meaningful context since consumer products have an average NPS of 23. This data indicates that your product is below the average ratio of likelihood to recommend, and should be improved in order to be able to compete on the market.




Solution Consultant QA & UX

Do you have questions about Quantitative Usability Study? Would you like to have more information on how we can help you make your digital products even better? Just send us a message, one of our Solution Consultants will be happy to get in touch with you.

Remote testing – Secure, fast, and fully flexible access to our global crowd

Your testing projects can be realized fully remotely and without physical interaction – our remote testing services offer a fast turnaround and make testers available across the globe.

@ Contact