Trust the AI. Then Fact-Check It Like a Journalist.

·
AdviceLesson

The AI you are building with is one of the most impressive tools ever made available to a person like you. It is also, on any given day, capable of being wrong with a full-throated confidence that would make a cable news anchor blush.

Both of these things are true at once, and living with both of them is the whole game.

The confident-wrong problem

Here is the pattern you will see:

You ask a specific question. You get an answer that sounds authoritative, well-organized, and completely convincing. Every word feels right. You act on it.

Later, you discover the answer was subtly — or catastrophically — wrong. A fact was invented. A function name doesn't exist. A quote was fabricated. A "well-known best practice" turns out to be an idea the AI made up on the spot.

This isn't the AI being malicious. It's the AI doing what it does: producing the most plausible-sounding response, whether or not it happens to be true. Plausibility and truth overlap most of the time. Not always.

The two categories to always double-check

Two kinds of things need verification, every time, no exceptions:

Facts. Specific numbers, names, dates, statistics, quotes, laws, medical or legal claims, anything you'd have looked up in a book. Do not trust these without checking. Ever.

Functionality. If the AI says the feature works, click the button yourself. If it says the form submits, submit the form. If it says the payment flow is correct, run a real transaction in test mode. AI is remarkably good at producing code that looks like it works and doesn't.

For everything in those two categories, adopt the journalist's rule: if your mother says she loves you, check it out.

What "checking" looks like in practice

You don't need a PhD to fact-check. You need a small handful of habits.

  • Independent search. Take the specific claim, drop it into a search engine, and see whether it holds up. Two sources beats one. A primary source beats two blog posts.
  • A second AI. Ask a different model the same question in a different way. If they agree, you're probably fine. If they disagree, dig deeper.
  • Actually run it. For features and code, always test. Click, type, submit, refresh. Trust your eyes over the AI's summary of what it just did.
  • The "how do you know?" question. Ask the AI to cite its source. Then check the source. About half the time, either the source doesn't exist or it doesn't say what the AI claims it does. That failure rate is important to feel in your bones.

The scary version: something breaks silently

The worst-case scenario isn't when the AI makes an obvious mistake. It's when it makes a quiet one — the kind that doesn't blow up until a real user hits it, days after launch.

Example: a form that stores data in the wrong field. Or an email that gets "sent" but never actually goes out. Or a permission check that looks right and lets anyone see anything.

You protect against these by:

  • Doing a "walk-through" of every important flow as a fake user before you ship it
  • Building small, so you always know what to test
  • Reading, or having the AI read back to you in plain English, exactly what a feature is doing — and comparing that to what you asked for

When to trust more, when to trust less

Trust the AI more on:

  • Explanations of concepts
  • Structuring your thoughts
  • Rephrasing, summarizing, editing
  • Suggesting options
  • Boilerplate code that has millions of examples in the wild

Trust the AI less on:

  • Anything specific to your business
  • Anything numerical
  • Anything legal, medical, financial, or safety-related
  • Anything about very recent events
  • Anything where being subtly wrong could hurt a real user

The healthy mental model

The best way I've found to think about it: the AI is a brilliant, tireless assistant who occasionally, with total sincerity, tells you the sky is green.

You wouldn't fire an assistant like that. They're too valuable. You just learn to double-check the sky-color claims.

The takeaway

Facts and functionality are not something to leave to a prompt. They are your responsibility, because they always were. The AI didn't change that; it just made everything else around it faster.

Trust the AI to help you build.

Then, quietly and consistently, check its work like your name is on the door — because it is.