Five lessons, about an hour and ten minutes. Module 1 named the reframe: you supervise the chatbot, you don't compete with it. This module is the mechanics of that supervision — what to hand it, how to brief it on your own knowledge base, and the specific habit that catches it inventing a policy that doesn't exist before a customer ever sees it.
A chatbot handling every ticket is a false economy — it will confidently answer things it shouldn't, and a founder who's automated away all human oversight finds out about the bad answers from an angry customer, not from a report. The job is choosing the split deliberately, not letting the bot's own confidence decide it.
A chatbot should handle volume. A person should handle anything where being wrong is expensive.
Hand it: factual, repeated, low-stakes questions straight from the documented knowledge base — shipping zones, ingredients, order status lookups. Always route to a human: anything touching money beyond a small threshold, any complaint with real emotion in it, any request the knowledge base doesn't clearly cover, and any customer who explicitly asks for a person.
Bot handles: "Does this ship to Kenya?", "Where's my order?", "Is this cruelty-free?" Always to Tolu: "I want a refund for the third time this has happened", any message that sounds distressed rather than just curious, and anything asking about a policy the help centre doesn't clearly state.
List five real support questions and sort each into "bot" or "always human," with one line of reasoning each.
A chatbot briefed loosely — "answer customer questions helpfully" — will fill any gap in its actual knowledge with something plausible-sounding, because plausible-sounding is what it's built to produce. The fix isn't a smarter bot. It's a tighter brief that tells it what to do when it doesn't actually know.
Point it explicitly at the documented knowledge base from module 6, and instruct it plainly: if the answer isn't in the documented material, say so and route to a human, rather than generating a plausible-sounding answer. This single instruction is the difference between a bot that's honestly limited and one that's confidently wrong.
"Answer customer questions using only the attached knowledge-base articles. If a question isn't clearly answered there, say: 'Let me get a team member to help with that' and flag it — never guess or infer a policy that isn't explicitly written down, even if it seems like a reasonable extension of what is."
A bot briefed against a stale or incomplete knowledge base will confidently repeat the stale answer, or invent something to fill the gap. Module 6's audit habit is what module 7's supervision actually depends on.
Write a briefing paragraph for a support chatbot, including the explicit failsafe instruction from the worked example.
Even a well-briefed bot occasionally states something confidently that isn't true — a return window that's wrong, a discount that doesn't exist, a promise about stock it can't actually verify. Catching this before it reaches a customer is a specific, checkable habit, not a vague sense of vigilance.
AI-drafted reply: "Since you're a loyal customer, I've applied a 20% loyalty discount to your next order automatically!" — no such discount or automatic-application system exists at Bumi Naturals. The three-way check catches this on step one: it doesn't trace to anything documented, because the bot invented it to sound generous.
Run the three-way check against an AI-drafted reply — real or made up — and note what you'd change, if anything.
Reviewing every AI-drafted reply as carefully as writing one from scratch defeats the point of using the tool at all. But reviewing so fast that "looks fine" becomes the actual standard is how a hallucinated promise slips through on a busy day. The skill is a fast check that's still a real check.
Read for the specific facts first — numbers, dates, promises — since these are where invention happens, not in the general tone or structure, which the bot is usually genuinely good at. A five-second scan of just the concrete claims catches most problems; reading every word at the same pace as the facts wastes time on the part that's rarely wrong.
Once you know to scan specifically for numbers, dates and promises rather than reading start to finish at one pace, the check gets faster, not slower — you're looking for the right thing instead of everything equally.
Time yourself doing a fact-first scan of three AI-drafted replies. Note how long it actually takes once you know what to look for.
An AI tool is a place information leaves the brand's own systems, even when the tool is being used entirely appropriately for a legitimate task. A customer's full card number, a home address paired with a name, health information mentioned in a return reason — none of this needs to be typed into a prompt to get help drafting a reply.
Replace a real name with "the customer," a specific address with "their delivery address," a health detail with "a personal reason" — the AI tool can draft an equally good reply from the anonymised version, because the wording doesn't actually depend on the specific personal details, only on the situation.
Instead of: "Sarah Johnson at 14 Cedar Close needs a refund because she's dealing with a medical issue and can't use the product" — paste: "A customer needs a refund; they've mentioned a personal health reason for returning the product. Draft a warm, brief reply approving the refund."
Take a message with a customer's real name and a personal detail, and rewrite it stripped, ready to safely paste into an AI tool.
Ten AI-drafted replies to real Bumi Naturals tickets are supplied. Three of them confidently state something that isn't true — an invented discount, a wrong return window, a promise about stock the bot couldn't have actually verified.
Whether the three-way check actually gets applied under the time pressure of ten replies, not just understood in the abstract as a good idea.
You should now have: a clear bot-versus-human split, a briefing template with a built-in failsafe, the three-way check, a fast fact-first review habit, and a confidentiality-stripping routine. Module 8 turns to the numbers this whole role gets measured by — and what gets lost when a metric is chased for its own sake.