Small and industry-funded supplement trials: how much to trust

A small or company-funded supplement trial is a weak reason to buy, but not a worthless one. Small studies are more likely to show a result that is just chance, and studies paid for by the maker tend to report friendlier results. The fix is not to dismiss them but to ask for a second, independent, larger study that points the same way.

Sales pages love the phrase "clinically proven," and behind it there is often a single trial with a few dozen people, paid for by the company that sells the product. This guide explains why size, funding and repetition matter, and how to weigh a trial like that without being cynical about it.

Why does the size of a trial matter?

The National Center for Complementary and Integrative Health (NCCIH) says that the number of people in a study can affect the outcome and what the results mean. Small samples may produce inconclusive results and are more likely to produce results due to chance, while large samples may give more reliable results and better reflect the wider population.

Think of flipping a coin ten times. Getting seven heads is not surprising. Getting 700 heads in 1,000 flips would be. A trial of 20 people can swing on a few individuals having a good or bad month, while a trial of 1,000 evens those swings out.

The statistician John Ioannidis made the same point in a well-known essay in PLoS Medicine. In his framework, a research finding is less likely to be true when the studies in a field are smaller, when effect sizes are smaller, when there is greater flexibility in designs and outcomes, and when there is greater financial and other interest and prejudice. Small, flexible, sponsored studies with modest effects hit almost every one of those conditions at once.

A tiny study can still be a real signal. Our Gorilla Flow review discusses a 2011 study in which 10 mg of boron a day lowered estradiol and raised free testosterone in 8 healthy men after one week. It is a real paper, but eight men for one week is a very small base to build a product claim on, and the review notes that the seller does not disclose how much boron, if any close to 10 mg, is in its formula.

Does company funding mean the results are wrong?

No, and it is worth being fair about this. Many good trials are paid for by the companies that make the products, because someone has to pay for them. Funding is a reason to read more carefully, not a verdict.

The evidence on sponsorship comes mostly from drug and device research, not supplements, but the pattern is worth knowing. A Cochrane review of industry sponsorship and research outcome (Lundh and colleagues, 2017) included 75 papers. It found that industry-sponsored studies more often had favorable efficacy results (risk ratio 1.27) and more often had favorable conclusions (risk ratio 1.34) than studies with other sponsors. The industry studies were not worse on most standard risk-of-bias checks, and they more often had low risk of bias from blinding. There was, however, less agreement between the results and the conclusions in industry-sponsored studies. The authors concluded that there appears to be an industry bias that cannot be explained by standard risk-of-bias assessments.

Put simply, a sponsored trial can be well run and still tilt toward a friendly reading. Supplements are a different field, so the exact numbers do not transfer, but the lesson does: look at who paid, who wrote the paper, and whether the conclusion goes further than the data.

The Joint Genesis review is a useful example of how we handle this. The label dose of the branded ingredient matches the trials the page cites, which we count in its favor. We also note that several of those studies involve researchers affiliated with the ingredient's manufacturer, and that a "10 times" figure on the page describes a laboratory measure, not a promised multiple of pain relief.

Why does replication matter more than any single trial?

The Federal Trade Commission (FTC) explains in its health products guidance that replication in a second study by independent researchers reduces the chance that the results of a single randomized trial were shaped by undetected, systematic bias, which can occur despite the best intentions of sponsors and investigators. An independent study that corroborates the findings gives much greater confidence in the first result. NCCIH agrees: a single study rarely provides a final, definitive answer, and repeating a study with different volunteers and investigators is how results come to be trusted.

The FTC guidance also shows what happens when replication fails. In one of its examples, a marketer touts a fruit drink as "proven to promote cardiovascular health" because a small study found a significant difference in arterial plaque compared with a placebo drink. A later, larger study found no significant difference, and so did a third large trial. The agency says the claim is unsubstantiated and that continuing to tout the earlier small study would be deceptive.

The same guidance describes a case where sponsorship is not a problem. In one example, a company sponsors a 100-person, double-blind, placebo-controlled trial, and several large independent European trials of a similar formulation and dose found meaningful results too. An independent expert confirmed the research met accepted standards, and the FTC says the evidence as a whole likely supports the claim. Funding alone did not sink it, because the result held up elsewhere.

What about a very big trial that says no?

Sometimes the best evidence is a large trial that cuts against the marketing. In the Prosta Peak review, the flagship herb is saw palmetto, and a Cochrane systematic review of 32 randomized trials covering 5,666 men found it was not better than placebo for urinary symptoms, including at double and triple the usual dose in a 72-week trial. The Protoflow review makes a related point: its lead ingredient is dosed at about a quarter of the standard studied amount, and the largest placebo-controlled trial found even higher doses ineffective for urinary symptoms.

Both examples show that a small positive study and a large negative one do not cancel out. When they disagree, the larger, better-controlled study usually deserves more weight.

What if the "trial" is the seller's own survey?

Not everything called a study is a trial. The Neuro-Balance Therapy review notes that the only evidence for that exact program is a self-run online survey of 100 self-selected people that was never published, peer reviewed or run with a control group. It relied on self-reported feelings, so it cannot show whether the routine caused the change or whether attention, activity and time did.

A quick test: does the study have a control group, random assignment and blinding, was it reviewed by a journal, and can you find it by searching its title? If the answer to most of those is no, treat it as a testimonial with a chart.

A practical checklist for small or sponsored trials

1. Find the number of participants and how long the trial lasted. Write both next to the claim.
2. Check for a placebo group, random assignment and blinding.
3. Read the funding and conflict of interest statements. Note whether the authors work for the maker of the ingredient.
4. Compare the ingredient, form and dose in the trial with the label on the product.
5. Search PubMed for other trials of the same ingredient. Look for an independent one and for any that found no effect.
6. Check whether the conclusion goes further than the results.
7. Ask what happens if the result does not replicate, and use the refund policy as your safety net.

None of this is medical advice. If you take medication or have a health condition, talk to your doctor or pharmacist before trying a supplement. To see how we weigh trials like these when we score an offer, read our how we review page.

FAQ

How many people should a supplement trial have?

There is no single number, because it depends on the size of the effect being measured. As a rule, larger trials give more reliable results, and a trial of a few dozen people should be seen as preliminary until a larger one confirms it.

Are industry-funded supplement studies always biased?

No. Many are well run, and the FTC guidance describes a sponsored trial that likely supports a claim when independent trials agree. Research on drugs and devices does show that sponsored studies tend to report friendlier results, so check for independent confirmation.

What does "replicated" mean?

It means a second study, ideally by an independent team with different participants, found a similar result. Replication makes it less likely that the first result was a fluke or an undetected bias.

Can I trust one positive study on the sales page?

One positive study is a starting point. Check who paid for it, how many people were in it, whether it matches the product and dose, and whether any independent study agrees or disagrees.