The fix wasn't more compute: how we cut merge-queue CI time in half

Every integration suite carries assumptions from when it was small, and most go unquestioned until the suite is large enough to make them expensive. CI is usually where that reckoning lands, and on a team that merges through a queue it lands hard: a slow suite stalls not just the engineer waiting on it but every pull request lined up behind them, so a rising median quietly becomes hours of collective waiting. The reflex at that point is to buy faster hardware, because compute is easy to purchase and an easy thing to blame. Sometimes that is right. This time it was not.

An all-green integration run that takes 27 minutes and still gets kicked from the merge queue looks like flakiness, but it is usually architecture. This spring our merge-queue integration tests slowed until that scenario was routine, and the fix turned out to be neither more parallelism nor a bigger runner. It was removing a single constraint that forced the whole suite to run serially, which cut merge-queue time roughly in half.

A merge queue serializes merges into main, running the full suite against the latest main before each pull request lands, so the gate is only ever as fast as the suite behind it. That makes the order in which you fix things matter more than the fix itself.

The pain

The suite hits a live backend rather than in-process mocks, and as coverage grew all spring the serial median climbed with it: about 4.4 minutes in February, 6.0 in March, 6.9 in April, 8.1 in May, and 10.1 by June. Even that understated the cost, because in a merge queue the tail governs behavior. Over a 30-day window the serial median was 9.8 minutes, but the p90 was 23 and the max 32, with 42 percent of runs over 12 minutes. It came to a head when a 27-minute all-green run was kicked from the queue anyway, and at that point runtime stopped being a metric and became a limit on how often anyone was willing to merge.

Name the bottleneck before you touch it

The reflex when CI is slow is to parallelize or move to a bigger runner, and both would have made this worse if we had reached for them first. The suite ran serially for one reason that had nothing to do with runner size: every test authenticated as the same shared user. Parallelize against a shared account and concurrent tests corrupt each other's state, whether that is the balance, the pods a test creates (the account's resources, not GPU Pods), the API keys, or the rate-limit budget, and the resulting flakiness gets blamed on the parallelism rather than the account underneath it. A shared test user is a scalability ceiling that stays invisible until serialization becomes the dominant cost, at which point it looks like a performance problem and teams reach for the wrong fix.

The sequence, and why the order was the point

The fix was two linked changes, and they had to land in this order.

Per-worker pool first. Each Vitest worker gets its own isolated test-user identity, so parallel files no longer fight over one account's state. This had to land first, because anything built on a still-shared account would corrupt data rather than run faster.

Split the long poles second. The suite runs about 480 tests across 63 files; we split the five largest into 22 self-contained files, moving 123 test cases byte-identical so behavior was preserved. The largest single file ran about 219 seconds on its own, a 3.5x outlier that set a floor under wall-clock time no matter how many workers ran. Splitting is a distinct problem from parallelizing, and the shared-state fix comes before both.

Validate the mechanism, not just the code

Passing tests and a trustworthy merge gate are different standards, and a canary is how you earn the second. Before gating any PR, we ran the same commit three times as a non-gating canary on one of our CI runners against a non-production stage, with the pool sized to match the fork count; holding the commit constant meant any variation had to come from the mechanism or the environment rather than the code.

The three runs landed in a tight 5.4 to 6.5 minute band, each at about 2.66x effective parallelism. That figure is measured, not theoretical: those same runs did 866 to 1040 seconds of cumulative test work in 326 to 389 seconds of wall time. Run 2 carried the signal. It reported five failed files, but only one was a real test failure: four files aborted at beforeAll on a single burst of 10 "fetch failed" network errors, the kind of infrastructure noise that would have hit the serial suite too, while the fifth held the genuine finding, an adversarial test that floods requests by design tripping a rate-limit rejection once several workers ran at once. Running the same commit three times is what separated that one real concurrency bug from ordinary network flake before the gate went live, where a single green run would have shipped it to production and let it pass as flakiness.

Four decisions worth carrying forward

1. Fix the architecture before adding parallelism, because reversed, the split only corrupts data.
2. Canary the mechanism rather than cutting over at once, since repeating one commit is what isolated the real bug from noise.
3. Size for utilization rather than raw speed. The suite is I/O-bound, spending most of its time waiting on network round-trips rather than on CPU, so you can run more workers than the runner has cores and still gain. Running more workers on our existing runner matched a larger runner's wall-clock time at far better utilization, so we avoided moving to bigger, costlier hardware.
4. Handle the adversarial test outside the pooled lane rather than weaken it. The rate-limit test floods requests by design, so instead of softening what it checks, it is excluded from the pooled run.

What flipping it on delivered

The canary and the production rollout are two different measurements, and conflating them would overstate the result. The canary, run non-gating, came in around 5.5 to 6 minutes against a 14 to 30 minute serial baseline on the same runner. Production is the operative figure: once the gate went live on 2026-06-24, the median dropped from 10.1 minutes to 5.2 across the first 6 runs and then to 4.5 across the first 19, about 2.2x faster and a 55 percent cut.

That is best read as an early signal rather than a settled average, since it rests on 19 runs measured against a 30-day serial baseline, and the number worth trusting is a like-for-like 30-day pool distribution that is still accumulating. Reliability moved along with speed, and in a merge queue the two are nearly the same thing: serial p90 was 23 minutes, every pool run so far has stayed at or under 9, and every PR queued behind a slow run waits for it too.

Across the 55 to 80 merge-queue runs in a typical week, that reclaims on the order of 5 to 8 engineer-hours of queue wait.

One wall remains. Consistent sub-5-minute runs now depend on the tests' own network round-trips to the backend rather than on worker count, so the next bottleneck is test quality, not parallelism.

Applying the pattern

This work lives in a private monorepo, so there is no repo to clone, but the pattern fits any parallel test suite blocked by a single shared account: give each worker its own identity, split the long-pole files before parallelizing, and canary on a repeated commit before it gates merges.

The wiring

The pattern is small once the identities exist. A pool-size variable drives both the number of seeded identities and Vitest's fork count, and the two must match. More forks than identities puts two forks on the same account and reintroduces the collisions you were removing.

// vitest.config.ts
import { defineConfig } from "vitest/config"

const poolSize = Number(process.env.TEST_POOL_SIZE ?? 0)

export default defineConfig({
  test: {
    fileParallelism: poolSize > 0,
    maxForks: poolSize || 1, // never exceed the pool
  },
})

A global setup step seeds poolSize isolated identities, mints a token for each, and writes them to a temp file the worker forks can read. Each fork then claims one slot deterministically from its worker id, so no two forks share an account:

// your pooled identity shape; carry whatever the tests need
type PoolUser = { slot: number; jwt: string }

function acquirePoolUser(pool: PoolUser[]): PoolUser {
  const workerId = Number(process.env.VITEST_WORKER_ID ?? 1) - 1
  return pool[workerId % pool.length]
}

Release is just cleanup: return the account's local state after the file or worker finishes so the next run starts clean.

Finding your pool-unsafe tests

Per-worker identities isolate everything scoped to a single worker's account, so what stays unsafe is anything that reaches past that boundary. Two shapes cover most of it: tests that assert on state shared across identities, and tests that deliberately push a shared limit to its ceiling, which is why the adversarial rate-limit test tripped the moment several workers ran. Exclude those from the pooled lane rather than weakening what they check; if you can isolate the shared state they touch, give them a dedicated path instead.

The win came from removing an architectural constraint, not from adding compute, and a suite that is serial by construction stays serial at any runner size.

Get Started

The best way to experience the ease of Runpod is with Flash.

• Try Flash: Runpod console
• Source code: github.com/runpod/flash
• Examples: github.com/runpod/flash-examples
• Docs: docs.runpod.io/flash/overview
• Discord: join us here

‍