Testing in the cloud versus real-life testing – two sides of the same coin

by | Nov 27, 2015 | QA, Trends in Software Testing, Usability & User Experience | 3 comments

When it comes to software testing, innovation is key. It’s hard to keep up with all kinds of trends popping up like mushrooms, and to keep an overview of the unique vocabulary. It has now been nearly two months since I started working at Testbirds without prior knowledge about IT or development, so this article can also be seen as a recap of what I have learned so far.

Crowdtesting was pretty easy to understand, as I am one of the end users of apps and websites myself and already had bad encounters with faulty apps. Our project TestChameleon however was quite a challenge for me, as it is deeply technical. But worry not, this blog post will stick to the basics and make automated testing with virtual machines a little clearer. In the following post, I will give you a short introduction to test automation and the most common buzzwords such as Agile Development, DevOps or Virtual Machines.
First, a little background info: As you are reading this blog, you are probably familiar with some parts of software testing, especially crowd testing. It’s pretty obvious that software needs to be tested before it’s released to the open market to avoid bad user experiences and functional problems. Perhaps you already tested a website for bugs and usability issues and know the importance of the feedback that comes from the user side.

Test automation with TestChameleon

However, when it comes to bug and compatibility testing, there is another approach that is getting more and more popular: Test Automation. Testbirds offers a service called TestChameleon (TeCh), which we have introduced to you in a post earlier this year. To keep it short: It’s a web service that creates every possible test infrastructure you could wish for in a virtual machine, similar to a chameleon that adapts its’ color to its’ current environment. Software testing in TeCh can be done manually or automatically, depending on what you need – but as the variety of operating systems, browser types, versions and add-ons in the market is changing almost daily, an automated test is the logical answer for getting feedback on the functionality and compatibility of the tested software.

Before getting into the benefits of test automation, we should probably talk about some development basics before. A rather new trend in development departments is called Agile or Lean Development. The base of Agile is a catalogue of methods and rules that ensure a more flexible development process as an alternative (and in many cases improvement) to traditional sequential or waterfall development. In Agile, the development activities are split up into so called sprints. For example, the waterfall method does all development steps consecutively, such as design, code or testing. Agile methods on the other hand run sprints with all those activities on all stages of the development process to find bugs and adapt to changes more quickly. It is a continuous improvement of all components of the product that is being developed to assure an early delivery.

The summary of agile methods can be found in the Agile Manifesto, which can be seen as a catalogue of essential rules for everyone who wants to adapt to that style of development.

Well, as one of the basic rules of agile development is the close collaboration of business people and developers, we come to the next term that was completely unfamiliar to me: DevOps. This is a combination of the words “development” and “operations”. DevOps is, like agile, more of a movement than a fixed term. The aim is to establish a flexible and productive working environment concerning the activities of building, testing and releasing software. So basically the barriers between operational organizers and the development department shrink to a minimum. Therefore, TestChameleon can be a part of the DevOps approach, as it enables continuous quality testing and is a helping tool in combining those two areas of work.

Virtual machines and Selenium

Next stop on the long way to intermediate TeCh knowledge would be: VMs – Virtual Machines. If you need to see how your website looks, for example, on one of those laptops with a fruit on its back, on the latest Firefox browser and using a strong add-blocker, you can create a virtual machine using TestChameleon. That means you can virtualize every device you would like to test your website on and control it from your own computer. As said before, the possibilities of different combinations of operating systems, browsers and add-ons are overwhelming. Test automation deals with those issues, as you can use programs like Selenium to create a script that runs automated tests. This method is so fast and flexible that you could test a website overnight on all common devices and by the next morning, you could start working on the bugs TeCh found.

At that point you might ask yourself: When do I use which testing method? What are the key benefits of each method?

First of all and most obvious – even though our cloud based TeCh can do quick and easy functional testing, there is clearly no way for automated testing to provide information about usability. This is and will be a task that the wisdom (and the humaneness) of the crowd is essential for. Exploratory testing is rather difficult with TeCh, as the tests run according to one script, which exactly specifies how and what will be tested. Also the crowd can provide us with a “dirty” testing environment. This means that several apps and components, which the person who wrote the testing script didn’t think of, will run in parallel.


Cloud and Crowd

Crowdtesting and automated testing should rather be seen as friends that are complementing each other. Automated Testing is mostly used for regression testing. That means that for example, if the development team implements a new feature to a website, regression testing can easily see if the feature works and if the rest of the website still runs flawlessly and remains unaffected by the change – on all kinds of devices. Crowdtesting is rather a real life test, often used shortly before release. Those two methods are no competitors, and using both testing techniques would let the client benefit from test that covers a multitude of quality issues. The basic principle: Cloud and Crowd work hand in hand.

I tried my best to explain a few of those words that are following me home for two months now after leaving the office. Sitting across our development team makes me realize that there is still a lot of stuff I don’t understand, but this also leaves us space for follow-up blog posts I guess. If you like, keep these facts in mind and they will probably lead you to a very agile conversation at the next IT Party. (Agile, ha-ha. Get it?)

Share it if you like it:

Tag Cloud

About the author

Franziska Rduch

Franziska Rduch

PR & Marketing Manager

Franzi writes articles, blog posts and whitepapers, does all kinds of research and plans events. In her free time she likes yoga, museums and dancing.



Other content that might be interesting for you:

Testbirds’ Healthy Computer Habits

Now-a-days, a majority of our waking hours take place in front of a computer. No matter the industry, a huge portion of the working force will find...

Get ready: Testbirds @ MWC 2020

Finally, we’re back - all set and ready for great things in 2020. And great things are already right on our doorstep as we’re starting off 2020 by...

Connected Cars: How 5G Influences Development

A lot of experts predict that the rise of autonomous driving and connected cars will be accelerating as 5G networks are rolled out worldwide. But...

Meet our Testers: Jason from China

This month we interviewed one of our most experienced testers in a new market for Testbirds - China. We are happy to introduce you to Jason, from...

Test Automation is Not Enough | “Adding the Human Factor” with Deutsche Telekom & Testbirds

Are you eager to learn which benefits crowdtesting adds to the development at Deutsche Telekom AG? Our webinar on July 23rd is exactly what you...

Celebrating the 1st Usability Speedtesting MeetUp at Testbirds

On September 4th, 2019 we hosted our first Usability Speedtesting MeetUp in our Munich office. Five companies, five test objects, five testers, and...

Why you need real people for Bug Tests in the Media & Broadcasting Industry

Bug tests in the broadcasting & media industry mean testing on different devices, screen resolutions, browsers and more. Therefore, you need...

It’s in the numbers: get your channel mobile-ready

Every quarter, the Dutch e-commerce organisation Thuiswinkel releases numbers to their market research regarding all things ‘home shopping’. It...

Testbirds wins Deloitte Technology Fast 50 Award

We are proud to call ourselves winners of the Technology Fast 50 Award by Deloitte, as one of the fastest growing Tech-Scale-Ups! We already knew...

Machine learning – The next big thing in QA and testing?

We at Testbirds believe that there are always new things to learn and keep your brains busy with. This is why we not only have a “Bird University”...

Testbirds specialises in the testing of software such as apps, websites and Internet of Things applications by using innovative technologies and solutions. Under the slogan, “Testing Reality”, the company offers various testing methods to its clients looking to optimise the user-friendliness and functionality of their digital products. With over 250,000 registered testers located in 193 countries, Testbirds is one of the world’s leading crowdtesting providers. In addition, the IT service provider utilises cloud based technologies to support customers in the optimisation of their digital products. The combination of the two testing methods delivers a unique and extensive portfolio that takes the quality of software to the next level. Testbirds was founded in 2011 by Philipp Benkler, Georg Hansbauer and Markus Steinhauser. Today, the company has over 100 employees. Other than its headquarters in Munich, there are now offices in Amsterdam, London and Stockholm, franchises in Hungary and Russia and sales partners in Italy.

© Testbirds GmbH. All rights reserved.

Subscribe to the Testbirds Whistler!

Receive updates on our innovative testing services, webinars, brand-new Nest features!

You have Successfully Subscribed!

@ Contact