The European Sensory Network ESN has conducted a research program to evaluate the role of virtual and immersive reality in sensory testing. The results show that preferences in the lab do not necessarily translate to the real or virtual world. The upshot: Immersive contexts can be meaningfully integrated into sensory testing. They afford reliable and economical substitutes and additions to real-life testing.

[You can also watch a five minute summary of the experiment on our youtube channel]

“We have seen a lot of hot new trends come and go. How can we be sure that this one is here to stay?” Ludovic Depoortere, board member of the European Sensory Network (ESN), raised this question during a talk he gave at the ESN meeting in Vienna in April 2016. It is a question that has been in the air over the last few years. However, it is now apparent that virtual reality (VR) is here to stay, and it is about to revolutionize every-day lives, from gaming to shopping. But what role can the new technology play in sensory research?

Amazing, but is it for real?

The warm April sun shines through the windows and into the meeting room of the hotel on the Schloßallee street in Vienna. But Fiorella Sinesio, sensory researcher from ESN member CREA, Italy, finds herself on a semi-dark stage surrounded by muscular men preparing to climb a rope for their next stunt in the Cirque du Soleil. She turns around and gazes down into the eyes of the transfixed circus audience. This is not the first time Fiorella Sinesio has used VR glasses. But with the technology constantly evolving, each time she slips the VR glasses over her head, that original feeling of amazement returns.

For the conference attendees who try out the VR glasses their reactions are much the same: first, they gape in astonishment, then they show an amused smile, and finally their bodies begin to move, watching from different angles as the circus artists start their climb.

The next day, a discussion is in full swing. With the help of virtual reality, product testers could easily be immersed in different environments and thus be able to sample products in a variety of contexts; with the snap of a finger, a product tester could sample a beer in a virtual bar, beach or living room. For decades, sensory scientists have been looking for more realistic settings for conducting consumer tests. Could VR glasses be the answer to their search by turning the laboratory into a real-life setting? “We can find out only if we conduct experiments that directly compare the different sensory research methods,” emphasizes Fiorella Sinesio.

Riette de Kock, chair of the ESN research, is standing in front of the attendees and asks: “Who’s in?”. The hands go up. The attendees of the meeting, all representatives of members and partners of the European Sensory Network, give the OK for a new ESN research program.

Sampling beer in Virtual Reality

In the months to come, the program is discussed, voted on, and then lead to a workshop with 400 participants at the EuroSense conference 2016 in Dijon. A cross-cultural, cross-laboratory research effort is initiated, and finally, an elaborately designed experiment takes place, and the results published. This, in a nutshell, is the general process of an ESN research program.

A VR artist recreated the pub “Trinity College” as a virtual scene. The product testers could immerse themselves, sample the beer and answer the questions without taking their VR glasses off.

The core team of “the immersive project” is comprised of ten people originating from six research centers/companies: CREA, Research Centre for Food and Nutrition (Italy), ADACTA (Italy), Biofortis (France), Haystack (Belgium), isi (Germany), and Firmenich (Switzerland). They are supported by ESN research manager Jean A McEwan. The project is funded by the European Sensory Network via the ESN member and partnership fees.

The Immersive team addressed the following question: Can a real-life product test – which is costly and time consuming and cannot be standardized – be simulated by a virtual reality product test in the laboratory? The design goes step by step, from a traditional lab test, through more and more immersive testing conditions, to testing in a real environment.

513 men are invited to participate in the study. They sample four beers and rate how much they like them. The testing takes place either in a laboratory or in a pub in Rome, the Trinity College. The researchers divide the men into five test conditions: A laboratory test (central location test, “CLT”), the real environment in the bar (“Real”), a test in an immersive room in which monitors depict the inside of the bar (“Immersive”), a Virtual Reality test in the laboratory in which the men wear VR googles depicting a 360 degree video from the bar (“VR 360”), and a VR test which uses a 3D-modell of the bar (“VR 3D”).

The research team decides to include two different ways of depicting the environment in the VR condition, VR 360 and VR 3D. “The 3D model has the advantage that it allows the product tester to act in the environment while wearing the VR glasses,” explains Fiorella Sinesio. “He can grab the glass of beer and fill out the questionnaire without taking the glasses off. This way, the product tester stays immersed in the scene during the whole testing.” On the other hand, the VR glasses with the 360-degree-video have to be taken off to drink and answer the questionnaire, which disrupts the immersive feeling.

In the real pub, product testers experience the true atmosphere and mingle with other people.

How real does it get?

Sensory researchers Fiorella Sinesio (CREA, Italy) and Silvia Abba (ADACTA, Italy) have had a couple of frantic weeks behind them, planning and organizing the immersive experiment. Now they are calm and concentrated as they oversee their team while it conducts the experiment at the bar in Rome. The men sit at a table; some talk with each other, others just take in the atmosphere of the bar with its wooden paneling, heavy tables, metal chandelier, and stained-glass windows. Soft music is playing in the background, the smell of beer and food is in the air, the conversations of customers at the bar filter through the room. It is September 2017, Rome is hot and the beer is a perfect eight degrees Celsius. Leisurely, the men sip the beer; afterwards, they fill out the questionnaire on a tablet in front of them.

Two weeks later, another testing takes place in a laboratory at CREA in Rome. This time, the product testers are in a sterile setting with white walls, white ceilings, neon light, and are divided from their neighbors by a partition. Via VR glasses, they are projected into a different environment. They see the same scene as the group in the real pub. When they move to the left, they see the stained-glass windows. If they look up, they see the wooden ceiling and the chandelier. They hear bar music and soft background noises. Will this “trick” help to bridge the gap between the need for standardization and the need for environmental contexts in sensory testing?

Waiting for the results. At the ESN meeting in Amsterdam 2018, Fiorella Sinesio presents the final data of the ESN immersive project.

More realistic results

It is April 2018. ESN is meeting in Amsterdam. Fiorella Sinesio enters the stage, 60 pairs of eyes are following her. It is the hottest day ever recorded in The Netherlands in April. In her calm way, the sensory researcher summarizes the results: “There was a tendency for the consumers to perceive a stronger difference between the beers in the laboratory compared to the more immersive conditions.” Thus, a beer that is preferred over others in a sensory lab might not be the favored beer in real life.

The results also show that a sensory test that involves choosing emotional words (CATA test) in association with the product sampling produces more positive words in the real and immersive situations compared to the laboratory. This finding is in line with another outcome: the consumers feel more engaged in the immersive contexts than in the laboratory. The VR 3D modelling and the immersive room come closest to the real bar in terms of product tester engagement.

Do the results encourage the use of VR in sensory research? Yes – they demonstrate that immersive contexts can easily be used in sensory testing, that they mimic real-world testing at a lower cost and with better standardization. Also, the new methods make us aware of the danger of translating a finding from the laboratory to real life.

During the EuroSense 2018 conference in Verona, ESN members present the results of the Immersive Project and initiate a lively debate.

It turns out that perceived product differences in a laboratory test environment may not be evident in the everyday world, and a product may need to be tweaked to stick out in the real world. Producers who wish to optimize their product and fulfill the expectations of their customers should integrate immersive methods into their toolbox. Certain sensory research questions may be better answered with VR, and there are times that an immersive room will best capture the consumers’ opinion.

The research team presents their results in an ESN workshop at the EuroSense conference in August 2018 in Verona. A lively discussion ensues during the workshop, and an important question arise from the audience: What is the added value of the new immersive techniques? Christelle Porcherot, co-author of the study takes over the microphone. “The results of specially designed VR studies already inform our research and development team at Firmenich, and the results influence how products are optimized.”

-----

The study was published online in May 2019 in Food Quality and Preference: Do immersive techniques help to capture consumer reality? Food Quality and Preference ( IF 3.652 )

DOI: 10.1016/j.foodqual.2019.05.004

Fiorella Sinesio, Elisabetta Moneta, Christelle Porcherot, Silvia Abbà, Lise Dreyfuss, Kévin Guillamet, Seppe Bruyninckx, Charles Laporte, Sven Henneberg, Jean A. McEwan

If you are interested in the publication, please send us an email.

Download text as PDF