The right answer looked evasive: an AI chatbot that separates its answers into four kinds

Summary

I made the chatbot that answers from my articles separate its answers into four kinds: what the sources actually say, interpretations derived from them, matters decided as answering policy, and speculation that is none of these. Alongside that, I published the text of those answering rules on the site as a public document that the answering AI itself can cite. In this article, I report, as a primary record, how I designed and verified these two changes.

It started with a single question a reader put to the chat: what happens when the source texts contradict each other, and can the chat even notice? The chat answered that when sources disagree, it adopts the newer statement as the current position. That answer is in fact the correct rule. It exists in the standing instructions passed to the model on every generation, that is, the system prompt, and it is something I decided and instructed in mid-July. But when the reader opened the citation number attached to that answer, the sources contained no such statement. Pressed further, the chat conceded that the wording was not in the sources and withdrew its previous answer, and when asked whether it was then closer to its own idea, it returned a vague answer with nothing to back it.

The cause was not that the chat lacked honesty, but that the structure of the instructions I had written was flawed. I rebuilt the instructions in two stages: for the version that introduced the four kinds, I ran 7 confirmation questions against the actual model, and for the version that turned the answering rules into a public document, I ran 8, before deploying to production. After that, I audited the way I write instructions for the AI from four angles, listed 76 items, and rewrote 19 of them from mechanical rules into statements of the meaning I want upheld and the reason why.

What I published is the text describing the policy, not the raw instructions. I still decline to disclose the instructions themselves.

In the body, I first introduce the four-kind rules and the current shape of the answers, then explain the design that reflects the answering rules from one source text into two surfaces. I then show the pre-deployment verification and measurements in a table, describe the defenses against instructions smuggled into reader input, and walk through how binding the model with mechanical rules failed twice in a row, along with the audit that swept every instruction of the same type. Finally, I describe the problems observed after moving to meaning-only instructions and the areas I have not touched, and close with three rules readers can carry into their own operations.

What you can take away

For anyone running a chatbot that answers with citations, this article covers the following four things.

• Separate answers into four kinds: have the model distinguish what the sources say, interpretations derived from them, matters decided as answering policy, and speculation that is none of these
• Turn the answering rules into a public document: reflect one source text on the site into both the reading surface and the chat’s knowledge base, and write the text under the constraint that paragraphs circulate on their own
• Convey meaning instead of binding with mechanical rules: after failing twice with sentence-ending rules and example phrases, audit 76 instructions and rewrite 19 into meaning and reasons
• Estimate the cost of moving to meaning: judge what to measure before making the move, from the voice breaking down on the real system after removing the rules and from measurements not yet taken

This article is my analysis, based on the primary records of revising and operating the chat that runs on this site.