How to tell if a journal cited by a supplement is credible

Check three things, in this order. First, look the journal up in the National Library of Medicine (NLM) Catalog and see whether it is currently indexed for MEDLINE. Second, search PubMed and the Retraction Watch Database to see whether the paper was retracted, corrected or flagged. Third, read the abstract and ask whether the study tested the ingredient, at the dose and in the kind of people the sales page implies. A journal can pass the first two checks and the paper can still fail the third.

This guide is general education, not medical advice. Talk to your doctor or pharmacist before you start or change a supplement. We also do not name any journal as predatory here, because that is a judgment about a publisher, and the tools below let you check for yourself.

Is a study credible just because it is on PubMed?

No. This is the most common mix-up. PubMed is a search tool, not a stamp of approval. NLM says PubMed holds more than 40 million citations and abstracts, drawn from several sources: MEDLINE, PubMed Central (PMC) and Bookshelf. MEDLINE is the part built from journals NLM has selected after review. PMC is a full-text archive that also includes individual articles collected to comply with funder policies, so an article can be in PubMed without its journal being a MEDLINE journal.

Look at the bottom of a PubMed abstract. Many records end with "[Indexed for MEDLINE]". That line means the record went through MEDLINE indexing. Its absence is not proof of a problem, because newer articles are still being processed, but it is a prompt to look closer.

How does a journal get into MEDLINE?

NLM states that it decides whether the scientific and editorial character and quality of a journal merit inclusion. At least two outside consultants, usually one scientist and one medical librarian, review each journal. They generally read articles from the last two years of content and judge them mainly on scientific and editorial quality. NLM adds that it views each journal comprehensively rather than using a fixed list of criteria. MEDLINE currently covers citations from more than 5,200 journals, so it is a selective set, not the whole publishing world.

To check a specific journal, open its NLM Catalog record and read the "Current Indexing Status" line. A record in the catalog on its own does not mean the journal is in MEDLINE or PMC, so read that line, not just the page title.

What is a predatory journal, and does it matter for supplements?

Researchers use the term for outlets that publish for the fee, with little real review or transparency. Defining it was hard enough that a 2020 study in BMJ Open convened a Delphi panel: 115 people took part in the survey and 45 experts finished the later rounds. They reached consensus on 18 of 33 items for a definition. The goal was practical guidance for institutions, funders and readers.

It matters for supplement marketing because a sales page has no way to say "this is a weak journal". It simply lists a title. Remember also that this works in both directions: not being in MEDLINE does not prove a journal is predatory, and a small regional journal can be honest. Treat indexing as one signal and read the paper.

A related trick is impact claims. A sales page may say a study appeared in a "high impact" journal. An impact number describes a journal's citation average, not the quality of one paper in it. If a page quotes one without naming the journal, you cannot check anything.

How do you find out if a paper was retracted?

Retractions are not hidden if you know where to look. PubMed has two publication types for this: "Retracted Publication" and "Retraction Notice". You can search them directly with retracted publication[pt] added to a topic, and a retracted paper's record links to its retraction notice. Check the paper's PubMed page for a "Retraction in" line.

The Retraction Watch Database is the second check. It is now openly available through Crossref, and it covers retractions plus corrections and expressions of concern, so limit a search to retractions if that is what you want. You can search by PMID or DOI, which makes it quick to look up the exact paper a sales page cites.

Citation of a retracted paper is a clear red flag, but silence is not proof of safety. A paper can be flawed without being retracted, which is why the third check matters most.

What does a real versus relevant reference look like?

A reference list can be made entirely of real papers and still say nothing about the product. Here are four patterns, each taken from our reviews:

• Real papers, wrong topic. The Oradentum review found that its "Scientific References" page, for an oral health capsule, links studies on a PDE5 inhibitor from Epimedium in a rat nerve cell line and on vitamin E for erectile dysfunction in aged rats. We opened the first two on PubMed and the titles match. Neither is about teeth or gums.
• A real study, credited to the wrong source. The Menopause Solution review notes that breast cancer risk figures are credited to the American Heart Association and the World Health Organization, while the landmark study is the NIH Women's Health Initiative trial, published in JAMA in 2002.
• On-topic research, missing dose. In the Vitrafoxin review, the cited mushroom papers are real and relevant, including a small human trial. But the product discloses no amount for its mushrooms, so the study cannot be matched to a serving. The trial itself, published in Phytotherapy Research, had 30 participants, and its record lists authors at a corporate mushroom laboratory, which is worth knowing when weighing it.
• A trial of a different formula. The BloodArmor review draws on a 12-week randomized trial in Nutrients of berberine, hesperidin and chromium picolinate added to metformin, a real and MEDLINE-indexed paper. BloodArmor's listed ingredients are berberine, gymnema, cinnamon, bitter melon, chromium picolinate and banaba leaf, with no hesperidin, so that trial tested a different combination.

None of these tells you the journal was weak. Each shows that credibility has to be checked at the level of the paper, and the paper against the product. See how we review for how we weigh this, and our blood sugar pages for more examples of cited research checked against a label.

A five-minute checklist

1. Find the paper on PubMed using the title, not just the sentence on the page.
2. Read the journal name and open its NLM Catalog record. Check "Current Indexing Status".
3. Look for a "Retraction in" line, and search the PMID or DOI in the Retraction Watch Database.
4. Read the abstract: human, animal or cells? How many people? How many weeks?
5. Compare the dose in the paper with the dose on the label.
6. Check who funded it and where the authors work. That is context, not a verdict.

FAQ

How do I know if a journal cited by a supplement is credible?

Check its NLM Catalog record for MEDLINE indexing status, read the paper itself, and see whether the paper tested the product's ingredient in people. No single check settles it.

Does PubMed vet the journals it lists?

Only partly. PubMed includes MEDLINE journals that NLM selected after review, but it also includes PMC and Bookshelf content, so being listed in PubMed alone is not proof of quality.

Where can I see if a study was retracted?

Search PubMed for the publication type "Retracted Publication" or check the paper's page for a retraction notice. The Retraction Watch Database, available through Crossref, also lists retractions, corrections and expressions of concern.

Is a journal outside MEDLINE always predatory?

No. Many legitimate journals are not in MEDLINE. Treat missing indexing as a reason to read more carefully, not as a ruling.