Select Page

Mystery Shopping – Unraveling the Mystery of Customer Experience

Nov 26, 2020 | Crowdtesting, Retail and E-Commerce

Customer loyalty, customer satisfaction, and service quality have always been vital factors in the retail and e-commerce sector. But there’s another aspect that’s becoming increasingly important: customer experience.

According to Review 42, 75% of shoppers prefer to buy from brands that personalize the shopping experience. 54% of companies also understand that customer experience (CX) is the most important priority.

But the question that arises is: What does the ideal customer experience actually look like? What do my customers expect, what do they reject?

It’s difficult to verify how the retail or e-commerce experience (whether in-store or online) is received by your target group, without the external help of testers.
How do I check whether my defined brand image and the perception of my customers match? What values and impressions does the layout of my store or the arrangement in my online store convey? Do my channels reflect the high quality of my offer and justify a high price point for my customers? How do I know if my delivery process works in Norway? Or whether my customers feel sufficiently informed about delivery and return conditions?

A simple customer survey is usually not enough to answer these questions. What you need is a deeper, unbiased evaluation of your target group in your target market. A popular and suitable way to get answers to the questions above is mystery shopping. But what is mystery shopping and isn’t it incredibly complicated? We have the answers.

What is mystery shopping?

Mystery shopping is a process in which test customers act out previously defined typical customer situations. Often, it’s a purchase in a store or online, but inquiries to customer support via different channels such as telephone, e-mail, or in person can also be part of mystery shopping.
Following the test purchase or test request, the testers’ impressions are summarized in written form and passed on to you.

Mystery shopping is a good opportunity to have your service or offer checked and evaluated by a relevant target group based on objective criteria. It’s crucial that the employees don’t know anything about the ongoing tests in order to get a realistic picture and not affect the results. It’s also conceivable to extend mystery shopping not only to your own company but also to competing companies.
Now that we know what mystery shopping is, I’d like to introduce you to the various possible applications.

Different forms of mystery shopping

The field of mystery shopping is very broad and strongly dependent on the objectives and the structure of the company for which the mystery shoppers are to work.

The simplest forms of mystery shopping are certainly in-store or online purchases. Here, for example, it’s possible to check how quickly the testers find a certain product, how the presentation in the store or on the website is perceived, and how the overall shopping experience is – well – experienced. What impression does the shop or online store give, is the product range perceived as high quality or substandard, are the products arranged in an attractive and sensible way, and so on.

Another form is the testing of things like vouchers in newspaper inserts or apps, customer loyalty programs, or customer guidance systems based on AI or chatbots for stores or online. The interaction of online and offline elements has become particularly important in recent years.

A third way to use mystery shopping is to check delivery times – optionally including returns management. Here, both the pure collection of data points like “How long does the delivery take?”, “When does the order & shipping confirmation arrive?”, “Does information about shipping and ordering come at all?”, “How long does it take until the company receives the return?”, as well as the subjective evaluation by the customer e.g. “Did they feel well informed?”, “Did the communication contain all important information?”, “Was the process simple?”, are possible.

Finally, there’s also the option of testing customer service. This can take the form of a fictitious complaint or inquiry, for example.
The test customers are instructed to ask certain questions via a specific channel or in person. The collected information is then used to compare ideal image and reality and to (re)train the employees if necessary.

Based on what you want to find out with a mystery shopping project and which area of your retail or e-commerce business you want to examine, there are different requirements for the test shoppers. This brings us to the next crucial point: tester selection.

The appropriate testers for your mystery shopping project

Once the plan for your mystery shopping project has been drawn up, i.e. once it’s clear which areas are to be tested and to what extent and once you have worked out the important evaluation criteria, possibly the most important question pops up: Where do I get suitable mystery shoppers?

Of course, this question depends strongly on how complex your product and your testing requirements are. If it’s just a matter of buying a carton of milk in the supermarket, you’ll easily find suitable mystery shoppers who will carry out the test for you for a relatively small fee.
But as complexity increases, so do the demands on your mystery shoppers.

The mystery shoppers need to correspond to your target group – for example, it wouldn’t (usually) be reasonable to have 70-year-old seniors test the purchase process in an online baby store or to have students go through a counseling interview for a house loan. Not only because the testers would find it difficult to assess the service, but also because the employee involved would immediately be taken aback.

In addition to the right demographics and interests, it’s just as important that the testers are thoroughly trained and embody their role as well as possible. A certain degree of acting performance and self-assurance is absolutely necessary. Especially in a personal consultation, the test buyer can’t just work his way along a script but must answer staff queries individually and spontaneously in case of doubt.

Last but not least, the preparation and follow-up of the test together with the testers, as well as an appropriate remuneration should not be disregarded. Even if these points may seem minor at first, the success of the mystery shopping project partially depends on exactly these factors.

The choice of testers and the associated effort are definitely among the sticking points in the implementation of a mystery shopping project. Time to focus on the strengths and weaknesses of this type of test.

Strengths and weaknesses of mystery shopping

Mystery shopping is an effective way to check the service quality of your store – whether offline or online – and to compare the customer’s perception with your own quality standards and brand image.
In contrast to customer surveys, mystery shoppers who correspond to your target group can give objective feedback and describe their impressions in much more detail than is possible on a ready-made form afterward.

In addition, a mystery shopping project can give you information about important data points regarding delivery and return processes, as well as the quality of customer service or advice. Thus, you can uncover potential for improvement that would otherwise have remained hidden.

These aspects help you build up customer loyalty and improve customer satisfaction in the long term. Only when expectations and reality are in harmony will customers keep coming back to you and not to your competition.

Nevertheless, there are also sticking points in the realization of mystery shopping. The execution of such projects requires careful preparation and professional implementation. This is the only way to ensure that you don’t spend a lot of time (and money) and end up with insufficient results.

Both the test setup and the selection of testers can be an almost insurmountable obstacle. It’s incredibly important that the testers actually correspond to your real customers and don’t have to slip into a role that is foreign to them. This is how you ensure that results aren’t distorted and, in the worst case, make changes based on false assumptions.

It’s also important that the employees who are part of the test do not notice anything about your project, but also that you don’t confront them with the results of the test buyers in a way that leads to a loss of confidence. Rather, you should bring the employees on board after the project has been completed and work together to develop suggestions for improvement instead of reproaching them.

Finally, it shouldn’t be neglected that you tie up the capacity of your employees while they take care of a mystery shopper or his order. During this time, they naturally can’t take care of a “real” customer.
However, all of these “weaknesses” can usually be well compensated by the improvements that can be made thanks to the results.


To achieve a consistent customer experience that’s in line with the values and goals you have defined, mystery shopping is a great way to match your ideas with the reality for the customer.

It’s advisable to seek support from companies that already have experience in the implementation and organization of mystery shopping projects and know the important stumbling blocks.
Especially when selecting testers that present your relevant target group, you shouldn’t make any compromises, because the success of your project stands and falls with the test buyers.

Only if your mystery shopping test is carried out professionally will you receive relevant results and be able to make targeted improvements.

If we have piqued your interest, please contact us. We have already carried out successful mystery shopping projects – even with very high-profile target groups, for example in very narrowly defined regions and niches.
Our experts are happy to develop a plan for your mystery shopping project together with you.

Share it if you like it:

Tag Cloud

Other content that might be interesting for you:

Hi from Testbirds Benelux

The first month of 2015 is almost over. Looking back on the past year, we can say that a lot of positive things have happened for Testbirds. As you...

Flashback Friday: MSN, our favourite instant messenger.

Do you sometimes reminisce about the age before smartphones, tablets and smartwatches? When was the last time you went through that box of old...

Melting ice cream test

Inspired by LadBible's legendary "Last Lolly Standing. Which Lolly Will Be The Last To Melt?" and as we are already melting in our offices during...

Round Table “Moderne Arbeitswelten 2.0 – Neue Arbeitsformen im digitalen Zeitalter”

Roundtable bei Testbirds Wie verändert das Internet unsere Arbeitswelt? Welche neuen Formen des (Zusammen-) Arbeitens ermöglicht die Vernetzung...

I am Franzi and this is my role at Testbirds

My Role at Testbirds is a series of blog posts that gives insights into the daily life at the office. Every so often, one of our employees will...

Testbirds Starting the New Year with a Kick-Off

All around the world, the New Year was welcomed with spectacular fireworks. Testbirds, however, welcomed 2014 with a big bang. What is a Bird...

It’s in the numbers: get your channel mobile-ready

Every quarter, the Dutch e-commerce organisation Thuiswinkel releases numbers to their market research regarding all things ‘home shopping’. It...

How to deal with the fragmentation of mobile devices

Mobile operating systems Today’s mobile landscape is more diverse than ever. In fact, as of 2015, there were over 24,000 Android devices on the...

Anti-Virus Software for your smartphone – Do you use it?

Remember the time we actually went and purchased a CD with Anti-Virus Software from our PC Reseller? They were not that cheap, but it had to be...

The Testbirds Party Bus is back in town!

MWC is one of the highlights for us at Testbirds every year. We love going there to network, to get updated about the upcoming mobile trends,...

Testbirds specialises in the testing of software such as apps, websites and Internet of Things applications by using innovative technologies and solutions. Under the slogan, “Testing Reality”, the company offers various testing methods to its clients looking to optimise the user-friendliness and functionality of their digital products. With over 250,000 registered testers located in 193 countries, Testbirds is one of the world’s leading crowdtesting providers. In addition, the IT service provider utilises cloud based technologies to support customers in the optimisation of their digital products. The combination of the two testing methods delivers a unique and extensive portfolio that takes the quality of software to the next level. Testbirds was founded in 2011 by Philipp Benkler, Georg Hansbauer and Markus Steinhauser. Today, the company has over 100 employees. Other than its headquarters in Munich, there are now offices in Amsterdam, London and Stockholm, franchises in Hungary and Russia and sales partners in Italy.

© Testbirds GmbH. All rights reserved.

Subscribe to the Testbirds Whistler!

Receive updates on our innovative testing services, webinars, brand-new Nest features!

You have Successfully Subscribed!

@ Contact