Multi-year testing of sports management games with real football fans
For over 3 years, more than 150 testers worked on a variety of tests and use cases for Sports Interactive across a range of devices such as PC, XBox, Nintendo, and smartphones.
Download Case Study
“Our commitment to elevating the gaming experience goes hand in hand with embracing innovative strategies. We choose Testbirds as our testing partner because their collaborative approach helps us shape strategic decisions, fine-tune the user experience, and harness crowd intelligence, enabling us to make informed decisions and elevate our games to new heights.”
Sports management games from Sports Interactive
Testbirds has conducted many tests for Sports Interactive over several years, covering many different areas of QA and UX. These have included playthroughs, player onboarding, accessibility, usability, prototype evaluations, bug tests, and performance tests. With the power of crowdtesting, Sports Interactive was able to make improvements and develop features that have taken their games to the next level and given their players an unparalleled experience in sports management simulation.
Provided clearer instructions of different controller functions.
Created more accessibility with additional color blindness modes.
Sports Interactive
Sports Interactive is one of the world’s leading developers of sports management simulation games and part of Sega Europe. Their games have been enjoyed by sports fans for 30 years, winning several awards and earning them a place in the history books of the gaming industry.
Client's needs
Over the course of three years, more than 150 testers have worked on a variety of tests and use cases for Sports Interactive, ranging from bug and performance tests to usability tests and user interviews.
In the variety of tests that have been carried out, there have been different requirements of what to test and in different games and game editions. The general aim has been to identify pain points and areas of improvement in order to create a seamless, intuitive, and enticing gaming experience – no matter what level of familiarity the player has with the game.
For Sports Interactive, testing is fundamentally important to the studio’s philosophy of ensuring high quality products are released. The studio believes that testing is not a one-dimensional process and therefore uses a variety of techniques, services, and technologies in order to understand the quality of their products and to build confidence ahead of releases.
In PC, console, and mobile games, the requirements of testing have similarities in their scope. Most often, the focus has been on the initial stages of the games where players familiarize themselves with the concepts, controls, and key elements of the sports management simulations.
“Usability studies often involve a lot of communication. Testbirds’ project managers connect with our teams to understand requirements for the project and then provide results and recommendations. Functional
testing, however, is a continuous process with ongoing two-way communication with Testbirds.
This helps us quickly address any issues. This back-and-forth has been really handy, especially for last-minute projects. Testbirds has been super flexible and accommodating.”
Test setup
As part of the QA, bug tests were set up to find functional issues that hinder or block the regular play of the games. . These could be simple bugs such as display issues, or more serious issues such as loading errors, connection issues or crashes.
To ensure a high-quality user experience, the usability of their games has been tested by players trying out the beginning of the games, giving feedback on their experience with tutorials and the ease of getting started. The accessibility of games has also been tested, focusing on how players with color blindness and dyslexia experienced the game both with and without accessibility improvements turned on.
Sports Interactive knew that many players were falling off games in initial stages, indicating that there were issues with getting players onboarded and engaged at the start of their gaming experience.
In order to dig deeper into these issues, players were required to play through the first one to two hours of the games (with some variation between projects) and provide feedback on their understanding of terminology and game concepts, their learning curve in tutorials, and which areas of the games they struggled with.
The initial feedback was followed up with more in-depth interviews with a selection of testers. The interviews aimed to find out more about the players’ experiences and why they did or did not continue to play the game.
The testers used were self-reported to be football fans of varying degrees who mostly had no prior experience with the sports management simulation games being tested. Some of them did however have experiences with other sports management or sports simulation games.
Most testers were male with age groups ranging from teenagers to mid-fifties. Most of the testers were geographically located in the UK, but there were testers located in other parts of the world who were also English speakers.
Devices tested on included computers, consoles, and mobile devices of different versions and models. The consoles tested on were different models of Xbox and Nintendo Switch, while smartphones and tablets consisted of both iOS and Android devices.
Show lessResults
Tutorials were detailed and extensive but also a bit overwhelming and too rigid at times. Testers found that parts of the onboarding went into too much detail for players who are not completely familiar and deeply invested in football tactics and terminology. These tutorials also did not allow skipping which left players feeling locked into the onboarding process.
For a more free and dynamic onboarding, the recommendation was made to allow skipping of tutorials and for new game features to be introduced gradually as players progressed through the game. Additionally, they were suggested to be simplified and shortened to decrease the amount of text-heavy and in-depth information, which was especially problematic for players with dyslexia, even when using accessibility improvements.
For those who got further invested in tactical and strategic choices, testers felt that it wasn’t always clear what consequences those choices led to. For example, in matches it wasn’t always clear what effect the formations or play styles had and throughout the season it wasn’t clear what effect coaching styles or transfer decisions had. Suggested improvements included further optional tutorials or information to increase their understanding of the effects of their choices.
Controller setups were also a topic of confusion for players as they found difficulties knowing when to use the analog stick or the D-pad. Another issue was players not discovering the cursor emulation function. This was improved with clearer instructions and more consistent usage of different controller functions.
Both the graphics and sound quality increased over the range of tests performed, as improvements were made to make matches more visually interactive, while the lack of sound effects and music were addressed.
When it came to accessibility, the three color blindness modes were helpful to players with only minor improvements needed for certain color combinations and contrasts between elements.
In terms of QA, there were only a few issues of low severity, such as smaller malfunctions and display errors. Sports Interactive also discovered potential performance improvements that resulted in better technical performance, for example with framerates and CPU utilization.
Show lessIn this ongoing and successful collaboration, Testbirds and Sports Interactive harness the power of crowdtesting to gain insights for enhancement across the entire gaming experience. This dynamic approach, coupled with a dedicated commitment to improvement, has helped Sports Interactive achieve consistently high quality standards. This partnership demonstrates the impact of crowdtesting in delivering user-centric innovations, cultivating player satisfaction, and ensuring the delivery of high-quality gaming products that resonate with a global audience.
“The usability reports from Testbirds as well as the data and interviews themselves, have been extremely useful to understand how players play our games, in particular those regarding specific accessibility requirements.
When it comes to quality assurance, having issues reported to us in real-time and pre-vetted by Testbirds have directly and positively impacted project decisions and future releases.”
Usability Testing
Discover how you can get real user feedback and finally meet your customers' expectations.
Find out more
We provide you with the latest insights from the world of crowdtesting
Stop guessing if your product meets your users' expectations and start making decisions based on facts.
