A message between AI agents was shown as sent, but it was not in the inbox

Summary

On 2026-08-19, in work on an initiative I am pursuing (research that rebuilds the development environment for AI agents from OSS components only), an instruction was given to an AI agent to send a message. The conversation took place on the screen of a desktop GUI app. The instruction asked the agent to send the message to the other agent with the message-sending tool (send_message) of an MCP server built in this research. This MCP server provides message exchange between sessions, and it writes a JSONL file for each destination agent name. This article calls this file the inbox file. The agent in the conversation was one connected to Qwen Code (version 0.21.13), one of three public OSS agent runtimes. According to the records of this research, Qwen Code called a built-in send tool with a similar name. The screen showed that the send succeeded, but the message had not reached any inbox file. This discrepancy was found by looking at the contents of the inbox files during manual work.

The core of the body is how a check script that judges message exchange between AI agents is built. The check script registers this MCP server with three agent runtimes, namely OpenCode, Qwen Code, and OpenHands CLI (versions 1.18.18, 0.21.13, and 1.16.0, respectively). It then judges whether message exchange succeeded with three checks: a send check, a receive check, and a control check. In none of the checks does the check script decide pass or fail from the AI agent’s own report. The result of the send check is decided by the lines in the other agent’s inbox file. The result of the receive check is decided by whether a marker string, generated at random for each run, appears in the output, and this string is not included in the prompt. The control check gives the same instruction as the receive check in a state where no message has been sent, and checks that the marker string does not appear in the output. On 2026-08-27, a tool call log was added. This log is a file to which the MCP server appends one line for each tool call. The check script also made it a pass condition that the IDs in the log match the IDs of the lines in the inbox file. The purpose is to distinguish cases in which the AI agent read or wrote the inbox file directly without calling the tool. In measurements with the CLI on 2026-08-19, 2026-08-27, and 2026-09-02, all three checks passed in all nine runs.

The body first describes the event in which the screen showed a successful send and the message had not arrived. Next, it describes how the check script that decides pass or fail by the lines of the inbox file and the marker string is built. It then describes how the judgment was strengthened with the tool call log, and the results of the measurements on the three agent runtimes. After that, it describes the conditions of the conversation in which the mix-up occurred, and an example from the previous day in which a report that child agents had completed disagreed with the records. It also describes the item added to the rule documents (instruction documents that AI agents read) after the observation. Finally, it describes the grounds for judging by records in a separate place rather than by output, together with where external research and official guidance stand on this.

What you can take away

For people who test collaboration between AI agents, I explain the following three things.

• The send check decides pass or fail by the lines in the other agent’s inbox file, and the receive check decides pass or fail by a marker string that is not in the prompt. The receive check is paired with a control check run in a state where no message has been sent. In the body, I show in a table the instructions and the pass conditions of the three checks
• Matching the IDs in the tool call log against the IDs of the lines in the inbox file lets you distinguish cases in which the AI agent read or wrote the inbox file directly without calling the tool. In the body, I also explain why this distinction is not complete
• All nine passes were measured with the CLI, and in the conversation in the desktop GUI app where the mix-up occurred, the tools of the MCP server did not appear. In the body, I describe a procedure to follow when registering an MCP server and confirming that it works: before giving an instruction, look at the configuration file and at whether the MCP server process started, and confirm that the tools are present in the conversation

The body is a discussion based on the records of work from 2026-08-18 to 2026-09-02 on the initiative I am pursuing, and on external materials checked on 2026-10-11.

The body names the products of the three agent runtimes. The content does not represent the views of any of their providers.

The send shown as successful, and the message that did not arrive

On 2026-08-19, in the work of this research, an instruction to send a message was given to an AI agent in a conversation in the desktop GUI app. The screen showed that the send succeeded, but the message had not reached any inbox file. An inbox file is a JSONL file that the MCP server built in this research writes for each destination agent name.

This MCP server provides message exchange between sessions. This article calls this MCP server the message-sending MCP server. The message-sending MCP server has the following three tools.

• list_agents returns a list of the other agents
• send_message sends a message
• check_inbox retrieves messages from the inbox file

Each time send_message sends one message, one line with the sender, the body, and an ID is added to the destination’s inbox file.

The agent runtimes used in this research are three public OSS ones, namely OpenCode, Qwen Code, and OpenHands CLI (versions 1.18.18, 0.21.13, and 1.16.0, respectively). ACP is a public standard for running agents from GUIs and editors. The name ACP is made of the initials of three words: Agent, Client, and Protocol. The agent in the conversation was an agent connected to Qwen Code through ACP.

The instruction was to send a message with send_message of the message-sending MCP server. Among the built-in tools that Qwen Code registers by default, there is a tool that sends messages. This article calls this tool the built-in send tool. This research has a document that records trial and error and a document that explains the integration with the desktop GUI app. The two documents say that Qwen Code called the built-in send tool, which has a similar name. The conversation records do not contain the name string of the tool that was called.

In addition to the success shown on screen and the message that did not arrive, the document that records trial and error also describes how this discrepancy was found. This discrepancy was found not by automatic judgment but by looking at the contents of the inbox files during manual work.

A check script that decides pass or fail by the lines of the inbox file and a marker string

The check script of this research decides pass or fail for message exchange by the lines of the inbox file and a marker string, not by the AI agent’s report. The check script is a Python script that registers the message-sending MCP server with the three agent runtimes and judges whether message exchange succeeded. This script was newly created in the work of this research on 2026-08-19. The check script runs three checks, the send check, the receive check, and the control check, in this order.

The check script generates a random string for each run. This article calls this string the marker string. The result of the send check is decided by the lines in the other agent’s inbox file. The results of the receive check and the control check are decided by whether the marker string, which is not included in the prompt, appears in the output.

The integration test from before 2026-08-19 checked only sending. This test was deleted in the PR that added the check script. The format for registering the MCP server was confirmed for each agent runtime with the output that its official registration command writes. The key in the configuration file is mcpServers for Qwen Code and OpenHands CLI, and mcp for OpenCode.

The check script launches a separate process as the other party in the checks. This process runs the same message-sending MCP server program as the AI agent uses, directly over JSON-RPC on standard input and output, without going through a model. This article calls this process the counterpart process.

The following table lists, for the three checks, the instruction to the AI agent and the pass condition.

The description at the top of the check script states the purpose of the control check. The purpose is to avoid passing the receive check when the marker string appears in the output without the inbox file being read. As such a case, the description gives the case in which the marker string output in the receive check leaked into memory or storage and remained there.

The check script does not pass states that it cannot judge. The states that it cannot judge are the following four.

• The agent runtime fails to start
• The run times out
• The inbox file does not exist
• The output is empty

If the destination is not registered, a call to the message-sending tool results in an error. For this reason, in the receive check, the AI agent’s name is registered as a destination of the message-sending MCP server only while the counterpart process sends. The process that registers this name is a different process from the counterpart process.

Strengthening the judgment with the tool call log

On 2026-08-27, pass conditions that match against the tool call log were added to the send check and the receive check of the check script. The tool call log is a JSONL file to which the message-sending MCP server appends one line each time a tool is called. Each line has the name of the agent that called the tool and the message ID, and it has no body. The code that writes this log was added to the message-sending MCP server on the same day, in the work of this research.

The following table lists the pass conditions added to the send check and the receive check on 2026-08-27.

The purpose of this strengthening is to distinguish message exchange through the tool from cases in which the AI agent reads or writes the inbox file directly without calling the tool. The check script also does not pass the check when there is no tool call log or when the tool call log contains a broken line.

The description of the check script says that this distinction is not complete. The reason is that the AI agent can read the location and the format of the tool call log with the same user’s permissions. An AI agent that can read the location and the format can also write both the inbox file and the tool call log itself, in the matching format.

Results of the measurements on the three agent runtimes

In the measurements on 2026-08-19, 2026-08-27, and 2026-09-02, all three agent runtimes passed all three checks. Each agent runtime was run once on each day, so there were nine runs in total.

The following table lists, for each measurement day, the judgment conditions and the number of agent runtimes that passed.

The three agent runtimes all used the same open-weight model from the same inference service.

All nine passes were measured with the CLI. In these measurements, the tools of the message-sending MCP server were available in each of the three agent runtimes. The mix-up on 2026-08-19 occurred in a conversation in the desktop GUI app. The tools of the message-sending MCP server did not appear in this conversation.

The document that explains the integration with the desktop GUI app contains a statement to the following effect. Even if the screen shows success, the send check does not pass when the message is not in the inbox file. The reason is that whether the send passes is decided by the contents of the inbox file. This statement is derived from how the checks are built, and there is no record of using the check script to reproduce the mix-up that occurred in the conversation.

Conditions of the conversation in which the mix-up occurred

In the conversation where the mix-up occurred, the tools of the message-sending MCP server did not appear in the conversation with Qwen Code connected through ACP. The two documents of this research say that Qwen Code called the built-in send tool. However, there is no statement that confirms whether the model chose the built-in send tool or Qwen Code resolved the tool by name. The scope checked was the document that records trial and error, the bodies of the two PRs from 2026-08-19, and the document that explains the integration.

This MCP server had been registered with the feature that manages MCP settings in bulk, on the settings screen of the desktop GUI app. With this registration, the connection check passed. However, no tools appeared in the Qwen Code conversation, and the MCP server process did not start either.

The two documents of this research describe Qwen Code’s built-in send tool as a tool with a similar name. The version of Qwen Code on the day of the observation was 0.21.13. According to a check on 2026-10-11, in this version’s tool name definition file and registration code, the built-in tools registered by default included send_message and list_agents. These two have names that are the same strings as the names of the tools of the message-sending MCP server. Qwen Code has a setting for a team feature, and these two built-in tools are always registered regardless of that setting.

The built-in list_agents returns a list of background agents. The built-in send_message has two modes: one that sends to team members and one that sends to background tasks. The words agent and task are the words used in the descriptions of the respective tools.

That the names matched is the result of checking the public source. It is not the result of confirming, in the conversation records, the name of the tool that was called in the conversation.

The Tools page of the Model Context Protocol specification, version 2025-11-25, has no provision for cases where a name overlaps with a tool built into the client. It also has no provision for cases where names overlap between different servers. This page says that each tool is uniquely identified by its name. It also says that tool names should be unique within a single server (SHOULD).

On the same day, message exchange from the desktop GUI app succeeded when the conversation was connected to OpenCode. In this conversation, the registration of the message-sending MCP server had been added to OpenCode’s configuration file. The agent in this conversation found a CLI session with list_agents and sent the marker string. A line whose sender was the agent in this conversation arrived in the CLI session’s inbox file. In the opposite direction, a Qwen Code CLI session sent a message to this conversation. The agent in this conversation read that message from the inbox file with check_inbox and output it on the screen.

An example from the previous day in which a report that child agents had completed disagreed with the records

On 2026-08-18, with an experimental parallel execution feature of Qwen Code, a report that the child agents had completed disagreed with the JSON record file that this feature writes. This feature launches child agents in parallel according to an orchestration script written by the model, and it writes a JSON record file for each orchestration run.

In the work of this research, this feature was measured in non-interactive runs. In seven trials there were 13 child agent launches, and none of the 13 completed. The child agents consumed 0 tokens, and they never called the model. In a re-measurement in interactive mode on the same day, two child agents completed. The run records of this research state the conclusion that the phenomenon of child agents not completing occurs only in non-interactive runs.

In the last trial, the model wrote the answer directly without launching any child agents, and in its final response it reported that three child agents had finished. In this trial’s JSON record file, the field for the number of attempted child launches was 0. The value of this field showed mechanically that the report was wrong.

The program built in this research that runs child agents in parallel always writes two counts in the run records of this research. The two counts are the number of attempted child launches and the number of child agents that returned a response body. The README of this program gives the false completion report of 2026-08-18 as the reason for making the two counts mandatory.

The document of this research that records trial and error classifies, as one type, failures in which a report of completion or success disagrees with a count field or with the contents of the inbox file. In this document, the successful send shown on screen on 2026-08-19 is the second example, following the completion report of 2026-08-18.

Articles from the same research that deal with the parallel-execution MCP tool and child agents include Once I wrote in the MCP tool description when to use the tool, AI agents called it and A failure that looked like a capability gap between coding agents was caused by how the API key was passed.

The item added to the rule documents after the observation

In a PR merged on 2026-08-20, one item with the same content was added to each of two rule documents. A rule document is an instruction document that AI agents read. The two are the default instruction document that AI agents always read and the skill document that they read when they use a feature that spans sessions. An AI agent created this PR.

The added item is an instruction for environments where the MCP server is not configured. The item instructs the agent not to substitute another tool with a similar name, to first check whether the tool exists, and, if it does not, to report that configuration is needed. As the reason, the item also states that a failure occurred in which a tool with a similar name was used and nothing arrived even though the screen showed success.

Grounds for judging by records in a separate place rather than by output

Both the report on 2026-08-18 that child agents had completed and the successful send shown on screen on 2026-08-19 disagreed with data that remains in a place separate from the report. The error in the 2026-08-18 report was confirmed with the field for the number of attempted child launches in the JSON record file that the experimental feature writes. The error in the 2026-08-19 display was confirmed with the inbox file. These facts show that it is better to decide the result of a check on agent collaboration by data that remains in a place separate from the report, rather than by the AI agent’s output.

The document of this research that records trial and error states what is judged by what, as follows.

• Whether sending a message succeeded is judged by the lines of the inbox file
• Whether something was executed, such as the launch of a child agent, is judged by counts written in the records, such as the field for the number of attempted child launches
• Whether something completed is judged by the deliverables
• The success shown on screen and what the AI agent reports in its text are not used for judgment

The receive check confirms that the output came from the inbox file by checking that a marker string that is not in the prompt appears in the output. The control check confirms that the same output does not appear when no message has been sent. When the two checks are paired, the control check can fail a case in which the marker string leaked into memory or storage, remained there, and appeared in the output without the inbox file being read. This property is derived from how the checks are built, and in the nine runs there was no case in which the control check failed.

The document of this research that records trial and error also describes a procedure to follow when registering an MCP server and confirming that it works. Before giving an instruction, look at whether the MCP server is written in the configuration file and whether the MCP server process started, and confirm that the tools are present in the conversation.

The idea of judging success or failure by the state and records left in the environment, rather than by the AI agent’s report, also appears in external research and official guidance.

According to the abstract of a study that measured false success (Advani, 2026), the failure in which an LLM agent claims to have completed a task even though the state of the environment does not show completion is called false success. One of the measured targets is 9,876 trajectories in which AI agents worked on tasks in tau2-bench, spanning eight model families. Whether each trajectory succeeded was decided without depending on the agent’s text. In the domains of tau2-bench that the study calls single-control, false success is reported to account for 45% to 48% of failures. The abstract does not define single-control. Judgment by an LLM (LLM judge) is reported not to have detected this failure well enough. AUROC is a metric that expresses detection performance from 0 to 1, and 0.5 is the same as chance. The judging configurations given in the abstract are five models, five prompts, and the full text of the task specification. In every configuration, AUROC on tau2-bench did not exceed 0.65. The abstract says that LLM judges rely on confident closing phrases and the volume of the action sequence, and that they do not look at verified changes in state.

Demystifying evals for AI agents gives an example of a flight booking agent. In this example, the outcome is judged not by whether the agent said it made the booking but by whether the booking exists in the environment’s SQL database.

The abstract of a study that detects hallucinations with tool execution receipts (Basu, 2026) proposes a framework for detecting hallucinations. In this framework, tool execution receipts are matched against the claims in a response. The receipts are signed with HMAC so that the LLM cannot forge them.

The limit of the check script was that, because the AI agent can read the tool call log with the same user’s permissions, there remains room for it to write the log itself in the matching format. These facts show that this limit is the same kind of problem as the forgery of records that the receipt study prevents with signatures. They also show that the approach of deciding pass or fail by data that remains in a place separate from the report is not a conclusion drawn only from the work records of this research.

Studies I referred to

The studies referred to in the body are the following two, and I checked their arXiv abstract pages on 2026-10-11.

• Laksh Advani, “From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents” arXiv:2606.09863
• Abhinaba Basu, “Tool Receipts, Not Zero-Knowledge Proofs: Practical Hallucination Detection for AI Agents” arXiv:2603.10060

Materials I referred to

The public materials referred to in the body are the following three, and I checked their original texts on 2026-10-11.

• Tools (the 2025-11-25 version of the Model Context Protocol specification) https://modelcontextprotocol.io/specification/2025-11-25/server/tools
• Demystifying evals for AI agents (Anthropic Engineering, 2026-01-09) https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
• tool-names.ts (the tool name definition file of Qwen Code v0.21.13) https://github.com/QwenLM/qwen-code/blob/v0.21.13/packages/core/src/tools/tool-names.ts