Repository navigation
copilot.exe fatal-aborts under load (BEX64 / 0xc0000409) — reproducible across 1.0.57 and 1.0.60 on Windows ARM64 #3687
Description
Summary
The interactive CLI process (copilot.exe) hard-aborts with a Windows fatal-app-exit instead of shutting down gracefully. It fires most reliably when several sessions start at once (Windows Terminal tab restore) under memory pressure, but it has also fired mid-session. The terminal has to be closed and reopened each time.
Over the last 5 days Windows Error Reporting logged 45 crashes of copilot.exe on this machine, across two CLI versions. This is reproducible and ongoing, not a one-off.
Likely the same root cause as #3250 (BYOK + parallel subagents) and #2335 (multiple hard crashes on Windows 26200). Same exception path and OS build. Filing separately because the trigger here is general load / simultaneous startup with the stock provider, and the multi-version data may help.
Environment
- Copilot CLI version:
1.0.60-0(also reproduced on1.0.57.0) - Bundled Node:
v24.16.0 - OS: Windows 11 Pro, build
10.0.26200 - Arch: ARM64
- Plan: Enterprise
- Host: Windows Terminal, pwsh
- RAM: 32 GB. Crashes correlate with free RAM dropping to roughly 2 GB.
Crash signature (Windows Application event log, Event ID 1000 / WER 1001)
Faulting application name: copilot.exe, version: 1.0.60.0
Faulting module name: copilot.exe, version: 1.0.60.0
Exception code: 0xc0000409 (STATUS_STACK_BUFFER_OVERRUN / __fastfail)
Exception Data (P9): 0000000000000007 (FAST_FAIL_FATAL_APP_EXIT)
Event Name: BEX64
IsFatal: 1
Fault offset: 0x0000000001cdd3dc
0xc0000409 with subcode 7 is __fastfail(FAST_FAIL_FATAL_APP_EXIT), the path the bundled Node/V8 runtime takes when it calls abort() on a fatal error (uncaught fatal, V8 fatal, or JavaScript heap out of memory). The faulting module is copilot.exe itself because the CLI ships as a Node single-executable, so V8 runs inside the binary. This is a deliberate runtime self-termination, not native memory corruption.
Two distinct WER fault buckets, one per version
| Version | WER bucket | Crashes (5 days) |
|---|---|---|
| 1.0.57.0 | 2212194956551539350 |
28 |
| 1.0.60.0 | 1899762302600568482 |
17 |
The 1.0.57 to 1.0.60 update changed the bucket but did not fix the abort path.
Crash clustering (auto-restart loop)
Crashes arrive in tight bursts, consistent with crash, auto-restart, crash again under the same memory pressure. Examples from one morning:
09:30:29, 09:30:36, 09:30:49, 09:30:53, 09:30:58, 09:30:58 (6 crashes in ~30s)
09:08:10 x4 (four sessions aborting in the same second)
Steps to reproduce (best-effort, load-dependent)
- On Windows ARM64, have several saved interactive sessions, each with a few MCP servers configured (mix of stdio node/npx and remote HTTP).
- Launch many at once, for example restore 6+ Windows Terminal tabs simultaneously, each running
copilot --resume. - While the MCP servers all cold-start and free RAM falls toward ~2 GB, one or more
copilot.exeprocesses fatal-abort with the signature above.
Not a guaranteed single-step repro, but the correlation with simultaneous startup and low free memory is consistent across dozens of occurrences.
What I ruled out
- Not a clean shutdown. The structured session logs that end with
[shutdown] Shutdown completeare headless prompt-mode runs, not the interactive TUI. The interactive process leaves no graceful log line, only the WER record. - Not MCP-config-specific. Reproduces after trimming to 7 global MCP servers and with an empty project-level config.
- Not a Windows Terminal hang. This is
copilot.exefaulting, captured by WER with a full dump reference.
Expected behavior
The runtime should not hard-abort under memory pressure or concurrent startup. At minimum the fatal reason should be recoverable from a log, not only from Windows Error Reporting.
Asks
- Surface the actual fatal reason before aborting. Today the only artifact is the WER record. If this is heap OOM, writing the V8 fatal / heap-limit context to the session log (not just terminal stderr, which dies with the process) would confirm it.
- If it is heap OOM, consider a higher default old-space limit for the bundled runtime, or backpressure when many MCP servers initialize concurrently.
- Guard against the crash-restart loop: if a session aborts on startup, avoid immediately relaunching into the same memory-pressure condition.
Attachments available on request
WER dump (WER.*.dmp) and the full per-timestamp crash timeline.
Activity
JW-Sthlm commentedon Jun 5, 2026
Update: this is NOT memory exhaustion. Stronger evidence points to a deterministic fault in the remote-export / request path.
After more data I need to correct my own framing above. The OOM/memory-pressure angle is largely a red herring. Three findings:
1. Identical fault offset across every crash
All 15 crashes logged on this machine share the exact same fault offset
0x0000000001cdd3dc(CLI 1.0.60.0). Heap-out-of-memory aborts crash wherever allocation happens to fail, so their offsets vary. A single constant offset across many crashes indicates one specific code path hitting__fastfail, not random heap exhaustion.2. Crashes with plenty of free RAM
The most recent cluster fired with 13.5 GB free of 32 GB and only 3 interactive tabs open. Memory pressure was not present.
3. Synchronized crash timestamps across independent processes
Crashes arrive at near-identical wall-clock seconds across separate
copilot.exeprocesses:Independent processes aborting at the same instant points to a shared timed trigger, not per-process memory growth.
Correlation: repoless remote export
The crashed sessions all run with
--remoteand the feature flagREPOLESS_REMOTE_EXPORT: true. The last log line of the backend before death is:The synchronized timing is consistent with a periodic remote-export flush (or an aligned interval timer in the remote path) firing across all sessions at once. All affected sessions are remote-steerable.
Repro refinement
--remoteenabled (repoless remote export on).Next step I am testing
Disabling
--remote(no repoless remote export) to confirm whether the crashes stop. Will report back. If maintainers can confirm whetherREPOLESS_REMOTE_EXPORThas an interval-driven flush, that would help localize the fixed offset0x1cdd3dc.Environment unchanged from the original report: Windows 11 build 10.0.26200, ARM64, CLI 1.0.60-0, bundled Node v24.16.0, Enterprise plan.
JW-Sthlm commentedon Jun 5, 2026
Decisive update: NOT remote export, NOT memory. Deterministic fault in the active request path.
Disabling
--remotedid not stop the crashes. WithNO_REMOTE=1set at user level and the dead sessions confirmed running withREPOLESS_REMOTE_EXPORT=false,copilot.exestill aborts at the same fixed offset0x0000000001cdd3dc. So both earlier theories are out:What the crashes have in common
The faulting sessions are resumed (
--resume) interactive sessions that were actively processing a model request when they died. Theirevents.jsonllast-write timestamp matches the crash second exactly. Sample (local time):Transcripts are modest (2 to 6 MB), so this is not a large-context issue. The structured session log ends mid-loop on lines like:
The constant fault offset across every crash (now 20+ on this machine, all
0x1cdd3dcon 1.0.60.0) indicates a single deterministic code path hitting__fastfail, not a resource/timing artifact. Multiple concurrent sessions raise the hit rate, which is why bursts of 4 crash in the same second, but a single active session also crashes.Dump incoming
I have enabled WER LocalDumps (full dump) for
copilot.exe. I will attach a minidump from the next crash so the offset0x1cdd3dccan be symbolized against your private symbols. If you can map that offset now, it likely points straight at the faulting function.Environment unchanged: Windows 11 build 10.0.26200, ARM64, CLI 1.0.60-0, bundled Node v24.16.0, Enterprise plan.
minny1205xx-cmyk commentedon Jun 5, 2026
JW-Sthlm commentedon Jun 16, 2026
Update from the same machine: root-caused this. It is not heap OOM, and there are actually two distinct failure modes that the original report blurred together. I hand-parsed nine of these minidumps (ARM64, so x64 dump tooling will not load them) plus the surrounding session logs.
It is not memory pressure
The crashing process's own telemetry at crash time:
v8_used_heapabout 52 MB against av8_heap_size_limitof about 4.3 GB (1.2 percent used), RSS about 150 MB, on a 32 GB machine. The0xc0000409with exception data0x7isFAST_FAIL_FATAL_APP_EXIT, a deliberateabort()from the bundled Node/V8 runtime. It is real, but it is not triggered by heap exhaustion. The original "free RAM drops to ~2 GB" correlation was a coincidence of load, not the cause.Failure mode A: the detached remote-export flush child aborts on shutdown
The processes producing these WER dumps are short-lived 4 to 5 thread children, not the interactive TUI. Each crashing PID matches a log line from its parent session:
These children are spawned during the shutdown dispose sequence (only when remote session export is active) to flush session events, and they
abort()instead of exiting. The full crashing call stack sits insidecopilot.exe(the Node single-executable); no MCP server or native addon is on the stack.The "clustering under load" in the original report is really one-flush-per-shutdown. A wrapper that repeatedly starts and stops
copilot --remotesessions produces one flush child per teardown, and on win32-arm64 each one aborts. Parallel teardowns produce the bursts. It correlated with "load" only because load meant more session churn.Confirmed workaround for mode A: run without
--remote. NoRemoteSessionExporter, so the detached flush child is never spawned, and the aborts stop. This isolates the fault to the remote-session-export path on win32-arm64.Failure mode B: the foreground TUI dies without restoring the terminal
This is the one that actually disrupts work, and the original report under-described it. When the foreground interactive process dies uncleanly (hard abort, or a non-zero engine exit), it does not run its terminal-restore path. The TUI enables mouse tracking (DECSET 1003 plus SGR encoding 1006) and focus reporting (1004) at startup and is supposed to disable them on exit. After an unclean death those modes stay on, so the shell is left emitting raw control sequences on every scroll, mouse move, or focus change, for example:
The terminal becomes unusable and the only recovery is closing and reopening every tab. Notably this mode does not always leave a WER dump, because a plain non-zero exit is not a fastfail.
Asks
abort().SIGTERM,SIGINT, and the fatal/abort path, so an unclean exit still sends the DECRST resets (mouse 1000/1002/1003/1006, focus 1004, bracketed paste 2004, alternate screen 1049). This would stop mode B from wrecking the terminal regardless of why the process died.Environment
Payload 1.0.63, bundled Node v24.16.0, Windows 11 build 26200, ARM64, Enterprise. The same flows on x64 here do not fastfail. Happy to share a parsed crashing stack and the exception record on request.
dimanardiyansah123-stack commentedon Jun 16, 2026
pgawascmlm.vip
JW-Sthlm commentedon Jun 21, 2026
Update from the same machine: root-caused this. It is not heap OOM, and there are actually two distinct failure modes that the original report blurred together. I hand-parsed nine of these minidumps (ARM64, so x64 dump tooling will not load them) plus the surrounding session logs.
It is not memory pressure
The crashing process's own telemetry at crash time:
v8_used_heapabout 52 MB against av8_heap_size_limitof about 4.3 GB (1.2 percent used), RSS about 150 MB, on a 32 GB machine. The0xc0000409with exception data0x7isFAST_FAIL_FATAL_APP_EXIT, a deliberateabort()from the bundled Node/V8 runtime. It is real, but it is not triggered by heap exhaustion. The original "free RAM drops to ~2 GB" correlation was a coincidence of load, not the cause.Failure mode A: the detached remote-export flush child aborts on shutdown
The processes producing these WER dumps are short-lived 4 to 5 thread children, not the interactive TUI. Each crashing PID matches a log line from its parent session:
These children are spawned during the shutdown dispose sequence (only when remote session export is active) to flush session events, and they
abort()instead of exiting. The full crashing call stack sits insidecopilot.exe(the Node single-executable); no MCP server or native addon is on the stack.The "clustering under load" in the original report is really one-flush-per-shutdown. A wrapper that repeatedly starts and stops
copilot --remotesessions produces one flush child per teardown, and on win32-arm64 each one aborts. Parallel teardowns produce the bursts. It correlated with "load" only because load meant more session churn.Confirmed workaround for mode A: run without
--remote. NoRemoteSessionExporter, so the detached flush child is never spawned, and the aborts stop. This isolates the fault to the remote-session-export path on win32-arm64.Failure mode B: the foreground TUI dies without restoring the terminal
This is the one that actually disrupts work, and the original report under-described it. When the foreground interactive process dies uncleanly (hard abort, or a non-zero engine exit), it does not run its terminal-restore path. The TUI enables mouse tracking (DECSET 1003 plus SGR encoding 1006) and focus reporting (1004) at startup and is supposed to disable them on exit. After an unclean death those modes stay on, so the shell is left emitting raw control sequences on every scroll, mouse move, or focus change, for example:
The terminal becomes unusable and the only recovery is closing and reopening every tab. Notably this mode does not always leave a WER dump, because a plain non-zero exit is not a fastfail.
This also affects VS Code's integrated terminal. In the latest repro the standalone Copilot terminal and the VS Code terminal both showed the same polluted state, while the editor/webview stayed alive. The VS Code terminal was also holding repeated MCP connect delay messages, which suggests the terminal host itself is getting contaminated once the Copilot session dies uncleanly.
Screenshot refs:
copilot-image-1b1fba.png: blocked model list until I signed in with the demo account.copilot-image-4996b7.png: VS Code editor still alive, but both terminal surfaces polluted after the Copilot session died.Asks
abort().SIGTERM,SIGINT, and the fatal/abort path, so an unclean exit still sends the DECRST resets (mouse 1000/1002/1003/1006, focus 1004, bracketed paste 2004, alternate screen 1049). This would stop mode B from wrecking the terminal regardless of why the process died.Environment
Payload 1.0.63, bundled Node v24.16.0, Windows 11 build 26200, ARM64, Enterprise. The same flows on x64 here do not fastfail. Happy to share a parsed crashing stack and the exception record on request.
patniko commentedon Oct 26, 2026
Thanks for the report. We've shipped fixes in this area since this was filed, and we believe this may now be resolved. Could you retest on the latest release (Copilot CLI v1.0.103) and let us know whether you still see it? If we don't hear back in a couple of weeks, we may close this issue — you can always reopen it.
Triaged as part of a Copilot SDK backlog review, with help from GitHub Copilot.