FutureDerm

The Fallacies of Comparative Studies

Share Article

 

Hmm, which side to take?
Hmm, which side to take?

When I refer to comparative studies, I’m talking about ones that evaluate two (or more) ingredients in terms of the same clinical characteristic, such as reducing inflammation and fading hyperpigmentation. There’s no doubt that these kinds of studies have their benefit: they quantify the relative efficacy of one ingredient to another, and are frequently used to demonstrate how a novel ingredient can be nearly or just as good as a tried-and-true one. However, it’s important to recognize that these results shouldn’t be taken at face value.

Fallacy #1: Lack of Vehicle Control

Talk about no vehicle control.
Talk about no vehicle control.

Most comparative studies only compare two different “active” or “featured” ingredients in two separate vehicles. So when you read that the two ingredients perform similarly, you accept that as the truth. In fact, I myself have done this multiple times in the past. And like I said above, it’s not wrong to do so. You just have to remember to not jump to conclusions.

For those of you who already spotted the potential caveat in this scenario, good job! For everyone else, the first source of error is that the two vehicles, in which the two “active” ingredients are suspended, have not been tested to see if they contributed to the final results; basically the studies are not vehicle-controlled. In order to eliminate any doubt, each (vehicle) would first have to be juxtaposed against its corresponding (vehicle + “active” ingredient) in terms of the same clinical characteristic as that in the comparative study. Then, the “active” formulation must demonstrate a statistically significant improvement (in the same characteristic) in comparison to its vehicle alone.

See how this can create feelings of doubt?

Lesson #1: Check and see if the two “sides” of any particular comparative studies have been vehicle-tested for efficacy.    

Fallacy #2: Different Mechanisms of Action

Benzoyl peroxide targets P. Acnes.
Benzoyl peroxide targets P. Acnes.

While the potential ramifications of not having studies with vehicle-controlled formulations are a very real issue, their overall effect on the validity of any comparative study and on their relevancy to real-life scenarios is small. Because virtually every study that’s accessible to the public on Pub Med originated from a published medical journal—meaning that it had to withstand close scrutiny by a large group of qualified individuals, it is unlikely (though still possible) that they all failed to notice the inclusion of a vehicular ingredient that has the ability to meaningfully skew the test results.

Therefore, it is this second “fallacy” that is of greater concern, which is that, the differing mechanisms of action between the “active” ingredients. Take this study for example. It suggests that 14% gluconolactone is about as effective as 5% benzoyl peroxide at reducing the number of inflammatory lesions (think papules and pustules) and non-inflammatory ones (think whiteheads and blackheads). Furthermore, the study concludes that fewer negative side effects were experienced by patients treated with gluconolactone. By reading this, many people will simply assume that gluconolactone is just the better treatment, and that the study proves that there’s no real need for the “antiquated use of benzoyl peroxide.”

Gluconolactone targets corneocytes.
Gluconolactone targets corneocytes.

However, what these people fail to recognize is that these two ingredients reduce acne via very different mechanisms of action. Benzoyl peroxide targets the acne-promoting anaerobic P. Acnes bacteria by flooding the skin with oxygen. Gluconolactone targets the excess dead skin cells (corneocytes) and forces them to fall off the skin. However, while benzoyl peroxide tends to penetrate deeply into the lower epidermis, gluconolactone tends to remain in the stratum corneum and the upper epidermis. Therefore, for deeper acne lesions such as cysts and nodules, benzoyl peroxide would be more effective.  The results of the comparative study reveal none of this. So again, take nothing at face value.

Think of it like this: One ingredient or person in this example can bend a spoon with his hands. Another person can bend a spoon with his mind. If you only looked at the results–bent spoons, you’d consider both persons to be pretty much equal, which is clearly not the case. Yes, that’s an extreme example, but you get the idea.

Lesson #2: Make sure that you thoroughly understand the mechanisms of action, because they can make a huge impact on your skin overall, depending on your needs.

Comparative Studies: Looking Ahead

Mhm. Let's sing Kumbaya and do it together!
Mhm. Let’s sing Kumbaya and do it together!

So keep in mind the two lessons discussed here as you peruse relevant studies, and/or when you see someone else cite a comparison study. Make sure to examine and question the appropriateness and correctness of the author’s claims and supporting documentation.

Finally, like I said before, comparative studies have many clinical and therapeutic benefits. However, it’s best to not try and look for the “best” ingredient for your skin/condition. As with anything, employ a multifaceted approach to treating your skin/condition by using ingredients that do the same thing, but via different mechanisms of action. So if you’ve got hyperpigmentation, use ingredients like retinoids, hydroquinone, niacinamide, and glycolic acid. If you’ve got acne, use ingredients like salicylic acid, retinoids, benzoyl peroxide, and azelaic acid.

Find this all confusing? Well, stay tuned because on my blog, I’ll currently writing a massive 30-part Page on what the Ideal Skin Care Routine is and how you can get there! Gosh, I sound like an infomercial… Anyways, here’s a sneak peek at how the Page will be organized.

 

You might also like

Product Review: Relastin Eye Silk

Accredited in [easyazon_link identifier=”0553383302″ locale=”US” tag=”cosmeticswiki-20″]The Skin Type Solution[/easyazon_link] by one of my idols, Dr. Leslie Baumann (director, Division of Cosmetic Dermatology and Assistant Professor of

About Myself

Nicki Zevola is the founder and editor-in-chief of FutureDerm.com. Named one of the top 30 beauty bloggers in the world by Konector.com since 2009, Nicki

#Mindey

@mindey