[shinharaguchi]

Description

A registered runner that is offline and not busy is now counted by the pool while it is within RUNNER_BOOT_TIME_IN_MINUTES. Today the pool skips the boot time check for any registered runner that is not online and idle, so a frequently running pool can launch extra instances while the first runner is still starting.

Root cause

With JIT config, the runner is registered in GitHub right after the EC2 instance is launched, before its agent connects. The example response of the JIT configuration endpoint and the observation in #3799 show such a runner as { busy: false, status: "offline" }.

countAvailableEc2PoolRunners treats a runner as booting only when it is not in the GitHub status map. A registered offline runner falls into the "not idle in GitHub and NOT counted" branch instead.

Change

Registered offline runners that are not busy get their own branch: counted while within the boot time, not counted after it. Online idle runners, busy runners and unregistered runners keep their existing handling. The check is based on runner status, so it also applies to non-JIT runners reported as offline within their boot time.

Logs

Pool size one, running every minute (instance IDs anonymised):

12:28:44 [pool] The pool will be topped up with 1 runners.
12:28:46 [pool] Created instance(s): i-aaaa
12:29:44 [pool] The pool will be topped up with 1 runners.
12:29:46 [pool] Created instance(s): i-bbbb
12:30:44 [pool] The pool will be topped up with 1 runners.
12:30:46 [pool] Created instance(s): i-cccc
12:31:44 [pool] The pool will be topped up with 1 runners.
12:31:46 [pool] Created instance(s): i-dddd
12:32:44 [pool] Pool will not be topped up. Found 1 managed idle runners.
12:35:24 [scale-down] Runner 'i-bbbb' will be kept idle.
12:35:25 [scale-down] AWS runner instance 'i-aaaa' is terminated and GitHub runner is de-registered.

Test Plan

• Added counts registered offline runners that are still booting. It fails on main and passes with this change.
• Split does not count registered busy or offline runners into a busy case and an expired offline case, and added an offline busy case.
• Ran yarn format, yarn lint, yarn test and yarn build in lambdas/. All passed.

Related Issues

Related #3799
Related #3809

[edersonbrilhante]

@shinharaguchi we need your commits to be signed, so it can be merged

[shinharaguchi]

@edersonbrilhante Thanks for pointing that out! I've re-signed both commits, and they now show as verified. Please let me know if anything else is needed.

[mcrawford-fms]

We run organization-level ephemeral runners with enable_jit_config = true and hit exactly this. With a scheduled pool, every run between the launch and the agent connecting saw the new runner as { busy: false, status: "offline" } and topped the pool up again, until runners_maximum_count. Today we work around it by invoking the pool Lambda from our own scheduler with a size corrected for registered booting runners; this change would let us drop that part of the workaround.

[shinharaguchi]

Thanks for confirming this, @mcrawford-fms! Good to hear it reproduces with organization-level runners and enable_jit_config = true as well.

@edersonbrilhante I checked that v8.1.0 still has the same pool counting behavior, so this fix still applies. If there is anything I can do to help get it into a release, such as rebasing, adding tests or updating docs, I'm happy to help.

[Copilot]

🟢 Approval recommended

No active review issues remain for the change to count booting offline runners toward EC2 pool capacity.

No open issues 🎉

What changed in this PR

This PR prevents unnecessary EC2 pool top-ups while registered runners are still starting, addressing the offline-runner behavior reported in #3799 and #3809.

Changes:

• Count registered, offline, non-busy runners toward pool capacity while within RUNNER_BOOT_TIME_IN_MINUTES.
• Add tests for booting, expired, and busy offline runners while preserving existing handling of other runner states.

File	Description
lambdas/​libs/​compute-providers/​aws/​ec2/​src/​control-plane/​pool.ts	Apply the boot-time check to registered offline, non-busy runners.
lambdas/​libs/​compute-providers/​aws/​ec2/​src/​control-plane/​pool.test.ts	Cover offline runners within and beyond boot time, plus busy-runner exclusions.

_{Review effort: Balanced}