An AI agent just got a 503 from your API

GitHub expects around 14 billion commits this year. Last year it was 1 billion.

Most of that jump is AI agents. Microsoft said the spike in agentic development "has tested our infrastructure's limits." On Oct 7, Actions, PRs and webhooks went down worldwide for about two hours.

I work on the platform side at browserless, and a lot of our users are teams running agents, so this one felt close to home.

Here's a normal agent tool call. It works in the demo. Then your API returns one 503, and it retries in a tight loop with no backoff and no limit, along with every other copy of that agent running at the same moment.

async function getOrder(id) {
  while (true) {
    const res = await fetch(`/api/orders/${id}`)
    if (res.ok) return res.json()
    // no backoff, no cap, ignores Retry-After
  }
}

A human gives up after two refreshes. An agent doesn't.

This is what I'd ship on any API that agents will call:

// 1. limit per API key, not per IP
app.use(rateLimit({ keyGenerator: (req) => req.get('x-api-key') }))
 
app.post('/api/orders', async (req, res) => {
  // 2. a retry gets the same answer, not a second order
  const idem = req.get('Idempotency-Key')
  const seen = await redis.get(`idem:${idem}`)
  if (seen) return res.json(JSON.parse(seen))
 
  // 3. tell it when to come back
  if (await overloaded())
    return res.set('Retry-After', '30').status(503).end()
 
  // 4. queue the work, don't hold the socket
  const job = await queue.add('create-order', req.body)
  await redis.set(`idem:${idem}`, JSON.stringify({ jobId: job.id }),
    'EX', 86400)
  res.status(202).json({ jobId: job.id })
})

1. Rate limit per API key, not per IP. In April, GitHub's own Copilot agent hit a rate-limiting bug that delayed 84% of new agent sessions.
2. Require an Idempotency-Key on writes. Agents retry on timeout, and a timeout doesn't mean the write failed. Without the key you get two orders.
3. Send Retry-After with every 429 and 503. Good SDKs respect it. For the ones that don't, your rate limit is the backstop.
4. Queue long work and return 202 with a job ID. Holding a connection open per agent is how you run out of sockets.

And keep a kill switch per API key. You'll need it at 3am.

We used to plan capacity for users. Now it's users times however many agents each of them runs.