Perry's Pen

A showcase for George Perry's writing

recent posts

about

How to Fact-Check AI: A Practical Guide to Spotting Truth from Fiction

How to Fact-Check AI: A Practical Guide to Spotting Truth from Fiction

AI-generated information sounds convincing, but that doesn’t mean it’s true. AI models are built to be persuasive—not always accurate. Here’s how to protect yourself from AI misinformation and spot the real facts.

Below, you’ll find actionable tips to quickly verify anything you read from an AI—so you can share and use AI-powered information with confidence.

1. Read Like a Pro: The “Lateral Reading” Technique

When we read a book, we read “vertically” (top to bottom). When checking AI, you must read “laterally” (across different tabs).1

  • Don’t stay in the chat window. If the AI makes a claim, immediately open a new browser tab.
  • Search for the specific claim, not the topic. If AI says, “Eating 30 almonds a day cures headaches,” don’t search “benefits of almonds.” Search “Does eating almonds cure headaches study.”
  • Compare sources. Look for consensus across at least three independent, reputable sources (e.g., a government health site, a major news outlet, and an academic institution)2.

2. Busted! Fact-Checking AI Quotes

AI often hallucinates quotes or attributes real quotes to the wrong people3.

  • Copy the quote. Take the specific sentence the AI provided.
  • Paste it into Google with quotation marks. e.g., “The only thing we have to fear is fear itself”.
  • Analyze results:
    • No results? The AI likely invented the quote.
    • Different author? The AI misattributed it.
    • Slightly different wording? The AI “paraphrased” but presented it as a direct quote.

3. Beware of “Ghost Links” and Fake Citations

AI models (like ChatGPT or Gemini) can generate realistic-looking citations that do not exist4.

  • Click every link. If the AI provides a URL, click it. Does it lead to a real 404 page? Does it lead to a relevant article or just to the website’s homepage?
  • Check the DOI. For scientific papers, ask for the DOI (Digital Object Identifier) and paste it into a resolver like doi.org. If the AI can’t provide a real DOI, the paper might not exist5.

4. How to Spot AI Hallucinations

Certain patterns in AI writing suggest it might be making things up. Be extra skeptical if you see6:

  • Vague Authority: Phrases like “Studies show…” or “Experts agree…” without naming the specific study or expert.
  • Perfectly Logical but Incorrect: The answer follows a logical structure (A + B = C), but the premise (A) is false.
  • Repetitive Hedges: If the AI apologizes excessively or uses phrases like “It is important to note” repeatedly, it may be masking a lack of concrete data.

5. Master the “SIFT” Method for AI Fact-Checking

This is a media literacy framework adapted for AI7:

  • S – Stop: The AI replies instantly. You should pause. Don’t use the info immediately.
  • I – Investigate the Source: Ask the AI, “What is the primary source for that specific statistic?” Then go find that source yourself.
  • F – Find Better Coverage: Is this “fact” reported by major outlets? If AI is the only one saying it, it’s likely false.
  • T – Trace Claims: Go back to the original context. AI summarizes; in doing so, it often strips away nuance (e.g., “Coffee causes cancer” vs. “Hot coffee above 65°C may increase risk…”).

Summary Checklist for Verification

StatisticsAsk for the year and source. Search the stat to see if it’s outdated.
QuotesSearch the exact text in quotes “…” to verify the author.
Legal/MedicalZero Trust. Consult a professional or official government database (.gov).
ImagesZoom in on hands, text in background, and shadows. Use Reverse Image Search.
CodeRun the code in a sandbox environment; do not copy/paste directly into production.

Sources:

  1. “Fact-checking AI with Lateral Reading – Artificial Intelligence (AI) and Information Literacy – Learning Guides at Jefferson Community & Technical College.” Jefferson Community & Technical College. 2023. https://jefferson.kctcs.libguides.com/artificial-intelligence/fact-checking-ai Accessed November 25, 2025 ↩︎
  2. “SIFT for Information Evaluation – Critically Evaluating Online Information.” Scottsdale Community College Library. 2025. https://library.scottsdalecc.edu/SIFT Accessed November 25, 2025 ↩︎
  3. Spinellis, Diomidis. “False authorship: an explorative case study around an AI-generated article published under my name.” Research Integrity and Peer Review 10 (2025). https://doi.org/10.1186/s41073-025-00165-z Accessed November 25, 2025 ↩︎
  4. “AI Hallucination Detector for Citations – Free Tool | SwanRef.” SwanRef. 2025. https://www.swanref.org/ai-hallucination-detector Accessed November 25, 2025 ↩︎
  5. Spinellis, Diomidis. “False authorship: an explorative case study around an AI-generated article published under my name.” Research Integrity and Peer Review 10 (2025). https://doi.org/10.1186/s41073-025-00165-z Accessed November 25, 2025 ↩︎
  6. Hufton, Andrew L.. “AI-generated research paper fabrication and plagiarism in the scientific community.” Patterns 4, no. 4 (2023): 100731. https://doi.org/10.1016/j.patter.2023.100731 Accessed November 25, 2025 ↩︎
  7. “SIFT – Empowering Informed Communities.” University of Washington Libraries. 2025. https://depts.washington.edu/learncip/sift/ Accessed November 25, 2025 ↩︎

Posted in

Leave a comment