AI is the smartest, dumbest thing you'll meet, so check its work
AI knows almost everything and still gets your facts wrong. Here's how we hunt for errors, stop the flattery, and teach AI not to repeat a mistake.
How do you check AI's work?
Assume something is wrong and go find it. Read every AI draft for facts, numbers, names and whose story it tells, and check anything you can't confirm against your own records. Then tell AI exactly what it got wrong so the next draft is better.
I work with AI all day, and the strangest thing about it hasn't changed. I said this to an If Possible customer in office hours in October:
it is the smartest, dumbest thing I've ever met. You know what I mean? Like it's this really bizarre thing.
It can write a strategic plan in minutes and then invent a button that doesn't exist in the software it's explaining. Both are true at once, so your job is to stay alert.
Why does AI make mistakes and make things up?
AI fills gaps with guesses. When it doesn't know a fact about your organization, it rarely stops to ask. It writes something that sounds right. It also learned from a world full of information that disagrees with itself, so confident and correct aren't the same thing.
Colleen pointed out that AI is trained on all the information in the world, and most of that information conflicts. Ask it to troubleshoot something and you'll eventually find yourself thinking, wait, that button doesn't even exist.
The deeper issue is that people expect AI to know everything:
Scott Williams · Recorded conversation, Sep 10, 2026there's a difference between knowledge and wisdom that, sure, it may have a ton of knowledge, it's the smartest thing on the planet knows everything but if you ask it for some wisdom. It's almost always going to fail you.
The fix starts before the draft. Colleen explained that a lot of AI slop happens when AI doesn't know the facts and makes decisions for you about how to fill the gaps. That's why our organization's AI brain marks missing information as "needs info" instead of letting AI invent it. Give AI the facts up front, and tell it to flag what it doesn't know.
Why do smart people stop checking AI?
Because AI writes so smoothly that our guard drops. After reading enough polished copy, it's tempting to think, this seems right, and move on. That's the moment errors slip into a grant report or a donor letter.
I compare it to walking down a dark alley. Your senses are up and you notice everything. Then there's sitting on the couch with someone you love, fully relaxed. Most people use AI in couch mode, and they should be in alley mode.
Colleen calls this one of the most important skills anyone using AI can build:
Colleen Cook · Recorded conversation, Sep 10, 2026you've got to stop and say, Do I actually understand what I'm reading? Do I agree with it? How do I know it's true
She also warned that AI can give you a false sense that you're an expert in a topic, right up until someone presses you on it. If you can't explain it without the draft in front of you, you don't understand it yet.
What should you check in an AI draft?
Check the things that would embarrass you if they were wrong: facts, numbers, names, quotes and claims about results. Then check the things that make it yours: whose story it centers, whether it sounds like you, and whether it's the right thing to say to this person right now.
Use this before anything AI touched goes out:
- Every fact and number. Dates, dollar amounts, names, titles, program results. Match each one to your source. If you can't find a source, cut it.
- The gaps it filled. Ask, "What did you assume that I didn't tell you?" Then answer or delete each assumption.
- Whose story it tells. One If Possible customer noticed in office hours that an AI draft about their services centered the organization instead of the people they serve. Good catch, and an easy fix.
- Your voice. Would someone who knows you believe you wrote it? A brand voice prompt gives AI something to match and gives you something to check against.
- The wisdom call. Is this the right message for this person at this moment? Only you can answer that.
For high-stakes documents like grant proposals, have a second person run the same check.
How do you stop AI from just agreeing with you?
Tell it to stop. State your goals clearly, ask it to be direct and honest, and ask it to argue the other side before you decide. Without that, chat-style AI tends to mirror you back, which feels good and teaches you nothing.
Colleen Cook · Recorded conversation, Sep 10, 2026it can create this, like, very sycophantic, like, echo chamber tendency in us. But if you allow it to be honest with you, because it's like reading data, it can be a mirror in a really helpful way.
Here's a prompt to copy into the start of any important conversation:
``` Before we start, here is my goal: [what you're trying to accomplish and for whom]. Be direct and honest with me. Do not tell me an idea is good unless you can say specifically why. When I share a draft or a plan, list its three biggest weaknesses first. If I'm missing information you need, say "needs info" and ask me instead of guessing. At the end, list every fact you used that I didn't give you. ```
What should you do when AI gets something wrong?
Say exactly what's wrong, ask why it made the mistake, and tell it not to do it again. Then save the lesson somewhere your AI will see it next time. One correction fixes a draft. Saved corrections fix the system.
That's what I told the customer who caught the organization-centered draft. Just tell it, "That's not community centric." It will agree. Ask why it went that direction, and it will explain its thinking. Then tell it not to do that again. I do some version of this every single day:
every single time I come back, the number one thing I'm hunting for is errors. You are wrong.
It can feel strange to train software this way. It works like a new employee who keeps asking where to submit receipts until one day they stop asking, because they learned. That's the whole idea behind treating AI like a new hire.
Can AI check its own work?
Partly. You can build a self-review step into any AI process, so it grades its own draft against your rules before you see it. You can also test it by asking about details you know you gave it. Both catch a lot. Neither replaces your read.
Colleen builds self-review into every new AI teammate while it's learning new work, because weak output eats your time. I like a simple test. In the same chat, say, "I'm testing you," and ask about something buried deep that you know you gave it. If it misses, you've found a gap before it cost you anything.
Over time you'll know your AI's habits, and as Colleen says, you'll know when the output is wrong. Until then, stay in alley mode. For more on keeping people in charge of what ships, read AI slop happens when AI sits in the director's chair and browse our implementation articles.
If you want more of what we're learning about working with AI, join the newsletter. And if you'd like to build these habits with Colleen, me and other nonprofit teams, get on the cohort waitlist.
Frequently asked questions
How do you check AI's work for errors?
Assume there's an error and go find it. Check every fact, number, name and quote against your own source, ask AI to list what it assumed, and read for whose story it centers and whether it sounds like you. For high-stakes work, have a second person check too.
Why does AI make things up?
AI predicts what sounds right. When it lacks a fact about your organization, it usually fills the gap with a confident guess instead of asking. Give it your real facts up front, and tell it to mark anything missing as "needs info" so it asks you.
How do I stop AI from agreeing with everything I say?
Tell it your goal, ask it to be direct and honest, and ask for the biggest weaknesses in your idea before any praise. Ask it to argue the other side. AI tends to mirror whoever it's talking to, so you have to invite honesty on purpose.
Can we trust AI for grant reports and donor communications?
Yes, with a human check every time. Let AI draft, then verify every number and claim against your records, and have someone who knows the donor or funder read it before it goes out. Save each correction so the same mistake doesn't come back.
Know an executive director who needs this?
