Who does a conversational AI's first person refer to? From a reader's comment to attaching speaker labels

Summary

Partway through a conversation, a reader pointed out that they could not tell whether the first-person boku (a Japanese word for I) appearing in the conversation referred to me or to the AI doing the answering. This site runs a chat where an AI that has learned my way of speaking answers with the articles as its grounds, and the comment above is one I actually received there. Prompted by this comment, I changed the system so that each utterance gets a speaker label before the conversation history is passed to the model. The excerpts passed in as grounds already carried labels by type of source, a countermeasure to a different mix-up that had happened earlier, and the speaker labels on the conversation extend that countermeasure to the conversation layer. As of writing, there are two kinds of labels for the conversation and four for the excerpts. There are four places where a countermeasure became necessary. First, who decided each rule in the documents that define the rules. Second, where the excerpts passed to the model as grounds came from. Third, who spoke each utterance in the conversation history. Fourth, which source the citation numbers attached to an answer refer to. In every one of these places, the AI and I had been putting sentences with different origins into the same sequence without a record of where each came from. That way of placing them was the cause common to all four.

That said, I cannot claim that attaching labels makes the model read origins correctly. According to a paper presented at ICML 2026, models judge where a text comes from not by the role labels they are given but by its style. This study, which measured role confusion, showed a result that flatly contradicted my hopes for the labels’ effectiveness.

In this article, I first lay out the four places where attribution got mixed, then explain how the labels on the conversation history and the excerpts are implemented. On top of that, I take up the first-person mix-up found through the reader’s comment and the case where citation numbers attached even to claims not in the sources, and describe how I came to have the AI speak in distinct modes: claims grounded in records, interpretations drawn from them, things decided as a way of answering, and things it does not know. Finally, I cover, in order, what I measured before deploying and what still has no numbers, the counterevidence against the labels and the limit the server cannot verify, where research on utterance attribution stands, and the obligation to let people know they are interacting with an AI along with the disclosure here, and I close with four rules that readers can apply to their own operations.

What you can take away

This article is for people running an AI that has learned a specific person’s way of speaking, and it explains the following four things.

• Counting up the places where attribution gets mixed: what happened in each of the four targets, namely who decided the rules, where the excerpts came from, who spoke in the conversation, and what the citation numbers point to
• Separating sources before passing them in: an implementation that labels the conversation history and the excerpts, and how to state in writing the order of precedence for what may count as grounds
• Designing the modes of speech: the path to having the AI distinguish claims grounded in records, interpretations drawn from them, things decided as a way of answering, and things it does not know
• Estimating the limits of labels: the 2026 finding that models judge origin by style, and the range the server cannot verify

This article is a study based on the primary records of running and reworking this site’s own chat.