Select Page

Mystery Shopping – Unraveling the Mystery of Customer Experience

Nov 26, 2020 |Crowdtesting, Retail and E-Commerce

Customer loyalty, customer satisfaction, and service quality have always been vital factors in the retail and e-commerce sector. But there’s another aspect that’s becoming increasingly important: customer experience.

According to Review 42, 75% of shoppers prefer to buy from brands that personalize the shopping experience. 54% of companies also understand that customer experience (CX) is the most important priority.

But the question that arises is: What does the ideal customer experience actually look like? What do my customers expect, what do they reject?

It’s difficult to verify how the retail or e-commerce experience (whether in-store or online) is received by your target group, without the external help of testers.
How do I check whether my defined brand image and the perception of my customers match? What values and impressions does the layout of my store or the arrangement in my online store convey? Do my channels reflect the high quality of my offer and justify a high price point for my customers? How do I know if my delivery process works in Norway? Or whether my customers feel sufficiently informed about delivery and return conditions?

A simple customer survey is usually not enough to answer these questions. What you need is a deeper, unbiased evaluation of your target group in your target market. A popular and suitable way to get answers to the questions above is mystery shopping. But what is mystery shopping and isn’t it incredibly complicated? We have the answers.

What is mystery shopping?

Mystery shopping is a process in which test customers act out previously defined typical customer situations. Often, it’s a purchase in a store or online, but inquiries to customer support via different channels such as telephone, e-mail, or in person can also be part of mystery shopping.
Following the test purchase or test request, the testers’ impressions are summarized in written form and passed on to you.

Mystery shopping is a good opportunity to have your service or offer checked and evaluated by a relevant target group based on objective criteria. It’s crucial that the employees don’t know anything about the ongoing tests in order to get a realistic picture and not affect the results. It’s also conceivable to extend mystery shopping not only to your own company but also to competing companies.
Now that we know what mystery shopping is, I’d like to introduce you to the various possible applications.

Different forms of mystery shopping

The field of mystery shopping is very broad and strongly dependent on the objectives and the structure of the company for which the mystery shoppers are to work.

The simplest forms of mystery shopping are certainly in-store or online purchases. Here, for example, it’s possible to check how quickly the testers find a certain product, how the presentation in the store or on the website is perceived, and how the overall shopping experience is – well – experienced. What impression does the shop or online store give, is the product range perceived as high quality or substandard, are the products arranged in an attractive and sensible way, and so on.

Another form is the testing of things like vouchers in newspaper inserts or apps, customer loyalty programs, or customer guidance systems based on AI or chatbots for stores or online. The interaction of online and offline elements has become particularly important in recent years.

A third way to use mystery shopping is to check delivery times – optionally including returns management. Here, both the pure collection of data points like “How long does the delivery take?”, “When does the order & shipping confirmation arrive?”, “Does information about shipping and ordering come at all?”, “How long does it take until the company receives the return?”, as well as the subjective evaluation by the customer e.g. “Did they feel well informed?”, “Did the communication contain all important information?”, “Was the process simple?”, are possible.

Finally, there’s also the option of testing customer service. This can take the form of a fictitious complaint or inquiry, for example.
The test customers are instructed to ask certain questions via a specific channel or in person. The collected information is then used to compare ideal image and reality and to (re)train the employees if necessary.

Based on what you want to find out with a mystery shopping project and which area of your retail or e-commerce business you want to examine, there are different requirements for the test shoppers. This brings us to the next crucial point: tester selection.

The appropriate testers for your mystery shopping project

Once the plan for your mystery shopping project has been drawn up, i.e. once it’s clear which areas are to be tested and to what extent and once you have worked out the important evaluation criteria, possibly the most important question pops up: Where do I get suitable mystery shoppers?

Of course, this question depends strongly on how complex your product and your testing requirements are. If it’s just a matter of buying a carton of milk in the supermarket, you’ll easily find suitable mystery shoppers who will carry out the test for you for a relatively small fee.
But as complexity increases, so do the demands on your mystery shoppers.

The mystery shoppers need to correspond to your target group – for example, it wouldn’t (usually) be reasonable to have 70-year-old seniors test the purchase process in an online baby store or to have students go through a counseling interview for a house loan. Not only because the testers would find it difficult to assess the service, but also because the employee involved would immediately be taken aback.

In addition to the right demographics and interests, it’s just as important that the testers are thoroughly trained and embody their role as well as possible. A certain degree of acting performance and self-assurance is absolutely necessary. Especially in a personal consultation, the test buyer can’t just work his way along a script but must answer staff queries individually and spontaneously in case of doubt.

Last but not least, the preparation and follow-up of the test together with the testers, as well as an appropriate remuneration should not be disregarded. Even if these points may seem minor at first, the success of the mystery shopping project partially depends on exactly these factors.

The choice of testers and the associated effort are definitely among the sticking points in the implementation of a mystery shopping project. Time to focus on the strengths and weaknesses of this type of test.

Strengths and weaknesses of mystery shopping

Mystery shopping is an effective way to check the service quality of your store – whether offline or online – and to compare the customer’s perception with your own quality standards and brand image.
In contrast to customer surveys, mystery shoppers who correspond to your target group can give objective feedback and describe their impressions in much more detail than is possible on a ready-made form afterward.

In addition, a mystery shopping project can give you information about important data points regarding delivery and return processes, as well as the quality of customer service or advice. Thus, you can uncover potential for improvement that would otherwise have remained hidden.

These aspects help you build up customer loyalty and improve customer satisfaction in the long term. Only when expectations and reality are in harmony will customers keep coming back to you and not to your competition.

Nevertheless, there are also sticking points in the realization of mystery shopping. The execution of such projects requires careful preparation and professional implementation. This is the only way to ensure that you don’t spend a lot of time (and money) and end up with insufficient results.

Both the test setup and the selection of testers can be an almost insurmountable obstacle. It’s incredibly important that the testers actually correspond to your real customers and don’t have to slip into a role that is foreign to them. This is how you ensure that results aren’t distorted and, in the worst case, make changes based on false assumptions.

It’s also important that the employees who are part of the test do not notice anything about your project, but also that you don’t confront them with the results of the test buyers in a way that leads to a loss of confidence. Rather, you should bring the employees on board after the project has been completed and work together to develop suggestions for improvement instead of reproaching them.

Finally, it shouldn’t be neglected that you tie up the capacity of your employees while they take care of a mystery shopper or his order. During this time, they naturally can’t take care of a “real” customer.
However, all of these “weaknesses” can usually be well compensated by the improvements that can be made thanks to the results.


To achieve a consistent customer experience that’s in line with the values and goals you have defined, mystery shopping is a great way to match your ideas with the reality for the customer.

It’s advisable to seek support from companies that already have experience in the implementation and organization of mystery shopping projects and know the important stumbling blocks.
Especially when selecting testers that present your relevant target group, you shouldn’t make any compromises, because the success of your project stands and falls with the test buyers.

Only if your mystery shopping test is carried out professionally will you receive relevant results and be able to make targeted improvements.

If we have piqued your interest, please contact us. We have already carried out successful mystery shopping projects – even with very high-profile target groups, for example in very narrowly defined regions and niches.
Our experts are happy to develop a plan for your mystery shopping project together with you.

Share it if you like it:

Tag Cloud

Other content that might be interesting for you:

Running a Marathon for the Evelina London’s Children Hospital

At Testbirds, we like to support our employees on a professional and personal basis, especially when they take on an amazing feat that helps make...

Black Box Testing – Enhancing User Experience

When you hear the term “black box testing”, what’s the first picture that comes to your mind? Let me guess - a black box? Most likely, my guess is...

Pre-App Store Rating – What, How and Why?

When developing an app it's often difficult for a developer to truly understand how and what end users want and require to have a great experience....

Webinar Recap | Test Automation & Crowdtesting – A Perfect Fit

In last week’s webinar with Magna, we introduced the V-Model 2.0 and explained how Crowdtesting and test automation complement each other...

Latest trends in the travel segment

As a frequent traveller, I always try to be on top of the latest travel apps to make life easier when preparing for my holidays. Remember the time...

Introducing Community Points: Showing Some Love to Our Testers

As a tester, you may have noticed some changes on your Dashboard this week. The biggest addition is a new point-system in our Nest: Community...

History’s Biggest Software Fails: The Heartbleed Bug

Previously on History’s Biggest Software Fails, we took the time to explore the World of Warcraft: Corrupted Blood Incidence. While the...

Will software test itself in the future? How Artificial Intelligence is changing the testing landscape

Whether you are steering your navigation system in your car by voice recognition or telling your smart speaker at home which song to play,...

Learn from our Bird Rob: Screencasts How-To

Need help capturing the screen of your PC or mobile device when reporting a bug? Our tester, Rob, is here to save the day! Hi, I am Rob Sep. I’m 53...

#Almedalen: Where politics, media and startups come together

“Almedalsveckan” (Almedalen Week). An 8-day political event in Gotland, Sweden that attracts 40 000 visitors. This is where all of Sweden’s most...

Testbirds specialises in the testing of software such as apps, websites and Internet of Things applications by using innovative technologies and solutions. Under the slogan, “Testing Reality”, the company offers various testing methods to its clients looking to optimise the user-friendliness and functionality of their digital products. With over 250,000 registered testers located in 193 countries, Testbirds is one of the world’s leading crowdtesting providers. In addition, the IT service provider utilises cloud based technologies to support customers in the optimisation of their digital products. The combination of the two testing methods delivers a unique and extensive portfolio that takes the quality of software to the next level. Testbirds was founded in 2011 by Philipp Benkler, Georg Hansbauer and Markus Steinhauser. Today, the company has over 100 employees. Other than its headquarters in Munich, there are now offices in Amsterdam, London and Stockholm, franchises in Hungary and Russia and sales partners in Italy.

© Testbirds GmbH. All rights reserved.

Subscribe to the Testbirds Whistler!

Receive updates on our innovative testing services, webinars, brand-new Nest features!

You have Successfully Subscribed!

@ Contact