Coke vs Pepsi Taste Test

Abstract

A number of friends (4) participated in a Coke vs Pepsi taste test, because I wanted to test their abilities to discern between Coke and Pepsi. Several tests were performed, testing the participant's ability to match a shot-glass of soft drink with its contents. Each test attempted to examine a different aspect of the subject's tasting ability, so several conclusions could be gleaned from this silly experiment. The main takeaways for the author are that:

• It's easy to tell the difference between Coke and Pepsi if you drink them regularly.
• It's not possible to tell whether Pepsi originated from a glass, can, or plastic bottle once the liquid has been poured from the original vessel.
• It's hard but not impossible to differentiate between variants of the same drink. (Coke vs Diet Coke vs Coke Zero).
• Almost all these findings are anecdotal because N=4 which is far too small for results to be at all conclusive.
• Doing this test is fun, and a worthwhile way to spend an afternoon.

Introduction

I grew up in a household that never really drank (except on special occasions) fizzy drinks. My girlfriend's family are constantly drinking Pepsi, and are lifelong, die-hard fans of Pepsi Max. At some point, someone boldly claimed that differentiating between Coke from Pepsi would be incredibly easy. I decided to go down the high-effort route of actually testing this hypothesis, and promptly bought ~10 different soft drinks + hundreds of small paper shot glasses and started to devise several experiments to subject my friends to over an afternoon. We cooked up different Pepsi themed foods, and made an afternoon of it.

Method

The experiment had a number of different rounds. Each round was conducted like so:

• Examiner pours soft drink into N numbered paper cups containing ~30ml of liquid each, and a piece of paper containing N circles drawn on it. Each circle is labeled with one the drinks that was to be used in that round. Which drink is poured into which cup is recorded.
• Which drink goes into which cup is randomized. (Stops people being like: "OMG cup number 4 was hard, I think it was Coke." when they leave the examination room.)
• The participant was called into a testing room with only the examiner.
• Participants could take as long as they wanted to taste each drink, and place each paper cup into one of the circles.
• When the participant was done, they left, and I compared the numbers on the cups to the labels on the circles.
• The next participant was called in.

Here several factors that could have influenced the results of the experiments, and what was done to try to control them.

• All drinks were drunk at room temperature. Participants argued energetically that this would affect the ability to taste them as most of their casual tasting is done at a chilled temperature. While I totally agree, we lacked the fridge space to refrigerate all drinks, and chilling some drinks and not others would give away information when tasting. (For example, if only Coke is chilled, the whole experiment is essentially meaningless as participants will easily be able to tell the difference based on temperature alone.)
• All drinks were opened within an hour of each other, otherwise the carbonation of the drink could yield extra information.
• Participants were not blindfolded. This might give some extra information away, but all drinks look the same to me so idk?

Tests

Test 1 - Intentional Deception Test

The first test was an "intentional deception test", where participants were asked to taste the following:

• Water
• Pepsi Max

and attempt to tell which is which. The idea for this came from the Ishihara eye tests for colourblindness. You've probably seen these before:

Ishihara plate 1. This plate obviously shows the number 12.

Ishihara plate 11. This plate apparently shows something but I can't see it because I'm colorblind.

A little-known fun fact is that Ishihara plate 1 (and 24) are control plates that everyone can see, regardless of whether you're colorblind or not! They're used to determine whether the person taking the test is intentionally failing it.

While I didn't really expect any of my friends to intentionally fail this test (because they were competing for the honor of the most attuned palette), one of my main takeaways from this endeavor was that doing something to familiarise the participants with the experiment before it gets serious is an excellent idea. This round, while essentially a joke, allowed me to practice and not make any mistakes when it mattered, and let the participants learn the process of the test.

I don't think I need any plots here. Everyone got the answer correct.

Test 2 - Coke Vs Pepsi

In the aptly named "Coke vs Pepsi" round, participants guessed between:

• Coke Zero
• Pepsi Max

And here are the results:

At this point, I should probably point out that Helen, Kirsten, and Lauren drink a fair bit of Pepsi. I would consider them connoisseurs of the black liquid. Julia does not, and probably represents the average person in this challenge. I should also mention that the general consensus when leaving this round was: "Get that weak shit outta here" so it's good to see the high confidence correlating to mostly correct answers.

Test 3 - Coke & Variants + Pepsi & Variants

In this mega-round, participants had the following drinks in front of them:

• Pepsi
• Pepsi Max
• Pepsi Lite
• Coke
• Coke Zero
• Diet Coke
• Coke Zero Vanilla

My hypothesis here was that while Coke and Pepsi alone would be relatively easy to differentiate, telling the difference between Coke Zero and Diet Coke might not. Also it should be mentioned that nobody here has drank Pepsi Lite before this challenge. Here are the results:

Interesting takeaways:

• Julia is at least consistent in her Coke/Pepsi mixup, fairly consistently saying Coke is Pepsi and Pepsi is Coke.
• Vanilla Coke isn't even a challenge. Upon entering the room, contestants would sniff every sample, and could guess the vanilla Coke instantly.
• Lauren only make a single mistake, and it being a mix-up between Coke Zero and Diet Coke is very impressive.

Given nobody got this 100% correct, I think it's a reasonable conclusion to make that it's VERY hard to correctly pinpoint all the soda variants, but it does seem that it correlates with skill, and how much soft drink you consume.

Test 4 - Pepsi Storage Method

This round had the following drinks:

• Pepsi Max (from a plastic bottle)
• Pepsi Max (from a glass bottle)
• Pepsi Max (from an aluminium can)

And here are the results:

From this limited sample, I can only conclude that nobody can tell what vessel Pepsi Max originates from. The results were worse than if selected at random! I recommend purchasing Pepsi Max in whichever container yields the cheapest $/ml.

Test 5 - New Drinks

This round had the following drinks to trial:

• Pepsi Max
• Diet Coke
• Que Cola
• LA Ice

Originally this round was supposed to be a mega-round featuring every unique drink in our possession. After the second round with 7 unique drinks, we decided that doing a mega-round would take ages, and not really yield any interesting information. This round features 2 classics, and 2 new drinks nobody has ever drank before. The only information that was given to them was that LA Ice contains sugar, and Que Cola is sugar-free. Here are the results:

From this, I guess we can conclude (again) that if you frequently drink Pepsi and Coke, it's very easy to tell them apart. Mixups between LA Ice and Que Cola make sense because nobody has tasted them before.

Conclusion

From these tests we can conclude a few things:

• It's easy to tell the difference between Coke and Pepsi if you drink them regularly.
• It's not possible to tell whether Pepsi originated from a glass, can, or plastic bottle once the liquid has been poured from the original vessel.
• It's hard but not impossible to differentiate between variants of the same drink. (Coke vs Diet Coke vs Coke Zero).
• Almost all these findings are anecdotal because N=4 which is far too small for results to be at all conclusive.

It is definitely worth mentioning that these findings are essentially anecdotal because it was done with 4 people, and with 4 people, pretty much any result can be ascribed to randomness. I'd also love to have produced some confusion matrices to show which colas are most often confused with each other, but you really need more samples before they start to show anything useful.

Crowning a Winner

What's the point of doing something if it's not to win, right?! To calculate a winner, I simply added together the total number of correct drink assignments and got:

This was good enough for us, because after demolishing round 3, we felt that Lauren was a deserving winner, and the three Pepsi drinkers clearly came out on top. However, one could argue that Kirsten mixing up LA Ice and Que Cola shouldn't result in -2 points, and following this argument probably leads us to conclude that not all soft drink selections are created equally. Being able to tell the difference between Coke and Pepsi in a test with two cups is easier than telling the difference between Coke Zero and Diet Coke in a test with 7 cups, and should therefore each correct guess should be weighted differently.

If I had far more samples, I'd love to use something like Item Response Theory to more mathematically crown the undeniable King/Queen of Soda, but since Helen and Kirsten didn't argue too much, it wasn't required.