In this webinar with Deutsche Telekom, we explored the pros and cons of crowdtesting versus test automation and shared how Testbirds became Deutsche Telekom’s trusted “external testing department.”

All topics of the webinar:

• How Testbirds became the external testing department of Deutsche Telekom AG
• Back-end vs front-end tests: different approaches, different testing methods
• Real-life example: testing the webmail app of Deutsche Telekom AG
• Panel discussion: crowdtesting vs. test automation – pros & cons

Developing a New Webmail App – Challenges

When Deutsche Telekom started developing their new webmail app, the biggest challenge was the huge amount of data they had to deal with.

We’re talking about 24M mailboxes, 3.6M unique users, 50M emails daily, 215M saved contacts, and 18M address book accesses.

With this huge challenge, Telekom began to restructure its organization in the development teams and moved from waterfall to agile development. While using the waterfall model, development took 12 months followed by a testing phase of three months. During this testing phase there were literally 30 people sitting in a room clicking through all the test cases in different browsers and with different e-mail clients.

So overall, the rollout took over a year – if everything went smoothly.

Switching to agile development, suddenly the project had to be split into sprints that took one month each. As they were used to, the team added a testing phase of 14 days after each sprint. Developing and testing in the same way as before, but just in smaller “packages”, was okay for the first halfyear as they developed a completely new product. However, after some time, the code and software became so huge and complex that an extra test phase for regression testing was needed after every sprint, adding two extra weeks.
Once again, development took a lot of time which was definitely not the idea when switching to agile development in order to release quicker and more frequently.  

Why Test Automation Wasn’t the Solution 

Deutsche Telekom was looking for a solution to make the test phase quicker and allow them to re-integrate test results into the sprint again. So, the idea was to automate the tests using the agile testing pyramid. That means that you usually test most functions with unit testing in the code, do a lot of integration testing, and only a small amount of manual end-to-end testing.  

What Deutsche Telekom did was to start automating the end-to-end tests because it seemed like a good idea (but in hindsight it wasn’t the best solution). Deutsche Telekom invested a lot of time and resources in automating those end-to-end tests and keeping them running – time and resources that were missing in other crucial parts of the development life cycle. Subsequently, they decided to outsource their testing projects and make development resources available again for their actual product development.  

Besides time and resources, Deutsche Telekom also had to deal with further challenges like device and sim card availability, a missing testing infrastructure, and the need for testers who match the target audience of the different products and needs.  

That was the point when Deutsche Telekom started to investigate crowdtesting and its capabilities. To find out what Deutsche Telekom had to say about the advantages of crowdtesting, like the big pool of devices, the fast test setup and execution, and the independent view on the application, watch the full webinar.