Summary
In some runs, when I ran the subtasks of an AI agent that solves coding tasks at the same time, the total time was longer than the sum of running them one at a time, even though every subtask finished. HTTP 429 rate-limit responses did not appear in the output of the harness, the program that calls the AI model to solve the coding tasks. Neither the number of finished subtasks nor the harness output showed that the total time had grown or that 429s had occurred.
With six subtasks at once, the total time was 280.1 seconds, while running the same six tasks one at a time took 225.0 seconds in total. In a run of 12 concurrent subtasks that each return a short sentence, 65 of 93 requests got a 429. When the last 300 characters of each subtask’s standard error and its error string in that run were searched, a rate-limit message was found for 0 of the 12 subtasks.
The six-subtask run did not record the HTTP status of each request, so I cannot tell whether any 429s came back or how many seconds the waits after them took. Each total time was measured only once, and I did not separate out the effect of the previous run. The records also cannot tell whether 429s would disappear if concurrency were limited to one or two subtasks.
What you can take away
This article is for people who build systems that run AI agent subtasks in parallel.
- A table that compares the total time of concurrent runs with the sum of one-at-a-time runs.
- How the median duration ratio of 8.21x was distributed across individual tasks.
- A table, split by elapsed-time interval, of the 429s that never appeared in the output, as observed in per-request records.
- The reasons the AI I develop with gave for not adopting a concurrency cap of three or staggered start times.
The article is based on the run records and design documents of a project I am working on.
Total time longer than the one-at-a-time sum even when every subtask finished
Together with the AI I develop with, I ran a program that solves coding tasks concurrently and measured the total time. In the September 2026 runs, the total time was longer than the one-at-a-time sum even though every task finished. In the October 2026 runs, HTTP 429 rate-limit responses did not appear in that program’s output.
The terms used in this article are as follows.
- Harness: a command-line program that calls an AI model and runs tools to solve a coding task
- Runner: a program I wrote that starts multiple subtasks and collects their results
- Subtask: a single job in which the runner starts the harness once and has it solve a task
- Heavy subtask: a subtask that solves a bug-fix or implementation task by calling tools repeatedly, so it sends many requests to the endpoint and their bodies are large
- Measurement program: a program separate from the runner that starts subtasks concurrently and measures the time they take
The table shows the results of running heavy subtasks with the measurement program, three at once and six at once.
| Concurrent subtasks | Total time | Sum of running the same tasks one at a time |
|---|---|---|
| 3 | 181.2 s | 100.8 s |
| 6 | 280.1 s | 225.0 s |
The six-subtask run was 55.1 seconds longer than the one-at-a-time sum. In both runs every task finished, so the number of finished tasks did not reveal the difference. Because running concurrently did not shorten the total time, the AI I develop with decided to keep the number of subtasks the runner starts at once at its default value of 1.
The 55.1-second difference compares single measurements. The three measurements, three at once, six at once, and one at a time, ran back to back without a pause, so I did not isolate the effect of rate limiting carried over from the previous measurement.
How to read the median duration ratio of 8.21x
For each task, I call the time it took when run concurrently divided by the time it took when run alone the duration ratio. In the run of six concurrent heavy subtasks, the median of this ratio was 8.21x. The ratios for individual tasks, however, ranged from 0.50x to 21.71x, so not every task slowed down by 8.21x.
Comparing by the number of concurrent subtasks, six at once has a larger median and a larger maximum than three at once.
| Concurrent subtasks | Median | Minimum | Maximum |
|---|---|---|---|
| 3 | 4.52x | 0.88x | 13.72x |
| 6 | 8.21x | 0.50x | 21.71x |
8.21x is the average of the middle two of the six tasks, and it is not the ratio of any single task. The one-at-a-time time was measured only once per task, so if that single measurement was an outlier, the ratio would move a lot, but I cannot check whether it was. The two September 2026 runs did not record the HTTP status of each request, so I cannot tell whether any 429s came back or how many seconds the waits after them took.
429s that do not appear in the harness output
When the measurement program searched the harness output, it found no messages about rate limiting or retries. The local proxy that records the HTTP status of each request between the harness and the endpoint, on the other hand, had 429s on record.
I call a subtask that returns a single short line and uses no tools a single-round-trip subtask. In an October 2026 run of 12 of these at once, all 12 completed only after receiving a 429 at least twice. A rate-limit message was found for 0 of the 12 subtasks. The search covered only the last 300 characters of standard error and the error string.
For the heavy subtask runs, the measurement program searched the full standard output and standard error for strings such as 429, rate limit, quota, retry, and backoff. None of these strings were in the output, either in the two September 2026 runs or in the October 2026 three-subtask run in which 9 requests got a 429.
429s from concurrent short requests and from single-round-trip subtasks
When the measurement program sent short requests to the endpoint concurrently without going through the harness, the runs with 6, 8, and 10 requests got no 429s, and the run with 12 requests got a 429 for 2 of the 12. Each request asked for the sum of 1 and 1, and the runs were spaced 120 seconds apart.
With the single-round-trip subtasks that return one short line, 429s and the total time grew as the number of concurrent subtasks grew. The values in the table are single measurements, each taken at least 120 seconds after the previous one.
| Concurrent subtasks | Total time | Requests | 429s |
|---|---|---|---|
| 3 | 21.8 s | 7 | 0 |
| 6 | 90.3 s | 22 | 6 |
| 12 | 157.6 s | 93 | 65 |
In the six-subtask run, the total time was set by the slowest subtask. One subtask that received four 429s took 90.2 seconds, while the four that received none finished in 6.8 to 13.6 seconds. The remaining 2 of the 6 429s went to one other subtask.
429s alone do not explain why the single-round-trip subtasks were slow. When I subtracted the total request time recorded by the local proxy from each subtask’s time, 13 seconds per subtask remained even in the three-subtask run with 0 429s. In a three-subtask run against a local mock server that waits 2 seconds and returns a fixed response, the remainder was 1.6 seconds. In the 12-subtask run, the longest request that got a 200 took 54.6 seconds. I do not know the reason for the gap between 13 seconds and 1.6 seconds, or why that request took 54.6 seconds.
This measurement does not reveal the length of the window the endpoint uses for rate limiting or its request limit. A 429 is a response that says a user sent too many requests in a given amount of time. It is defined in RFC 6585.
Per-request records for three heavy subtasks
In an October 2026 run of three heavy subtasks at once, all 11 requests sent within 9 seconds of the first request got a 200, and the first 429 came 13 seconds in, on the 12th request. When the 24 requests are split by elapsed-time interval, all 9 of the 429s fall between 13 and 22 seconds.
| Time since the first request | Requests sent | 200s | 429s |
|---|---|---|---|
| 0 to 9 s | 11 | 11 | 0 |
| 13 to 22 s | 11 | 2 | 9 |
| After 22 s | 2 | 2 | 0 |
In this run, all three tasks finished 85.0 seconds after the first start. In the array bug-fix task, the harness received four 429s by 16 seconds in, then sent no requests for 61 seconds, and got a 200 on a request at 77 seconds.
The sum of running the same three tasks one at a time was 100.8 seconds, longer than 85.0 seconds. That sum, however, comes from measuring each task only once in September 2026. The string-processing task took 74.5 seconds when run alone and 28.0 seconds in the October 2026 three-subtask run. Because the time for the same task varied this much, the AI I develop with judged that the difference from 100.8 seconds does not show that the run got faster.
A concurrency cap of three and staggered start times
Since 429s appeared even with three heavy subtasks, the AI I develop with did not adopt a rule capping concurrency at three. It also did not implement a mechanism that delays each start by one second. The reason was that the first 429 came 13 seconds in rather than right after startup, and the AI expected that staggering the starts would not change the spacing between requests after the 13-second mark. This expectation was not measured.
Of the 24 requests from the heavy subtasks, 21 had large bodies of 105 KB to 113 KB, but this measurement did not isolate whether the endpoint’s limit depends only on the number of requests or also on the amount of tokens. I did not measure runs of only one or two concurrent heavy subtasks, so I also do not know whether limiting concurrency would make the 429s go away. The HiveMind paper evaluated a method that uses an HTTP proxy to coordinate contention among agents sharing the same API, mainly against a mock API server, so its results cannot be compared with the values in this measurement.
The unsupported claim in the first documents that blamed the endpoint
After the September 2026 measurements, the AI I develop with stated as fact in three documents that the endpoint was the cause of the slowness. A first-pass review that followed pointed out that the evidence was insufficient, so the AI revised the three documents to say that the records cannot determine the cause. In response to the same finding, the AI also changed the measurement program so that it stops without making a judgment when it cannot read the harness output.
Materials I referred to
- RFC 6585 Additional HTTP Status Codes https://www.rfc-editor.org/rfc/rfc6585.html
Research I referred to
- HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads: The main evaluation was done against a mock API server, and the mock agents stop on unrecoverable errors. In the measurements in this article, every subtask finished.