Skip to content

fix: keep systemd-firstboot running headless (detach stdio from console) - #279

Open
guysoft wants to merge 1 commit into
develfrom
fix-firstboot-mask
Open

fix: keep systemd-firstboot running headless (detach stdio from console)#279
guysoft wants to merge 1 commit into
develfrom
fix-firstboot-mask

Conversation

@guysoft

@guysoft guysoft commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Problem

On Raspberry Pi 5 (Debian trixie, kernel 6.12.47) a flashed image with an
uninitialized machine-id freezes on first boot: systemd-firstboot.service
starts its wizard, the child opens /dev/console and the boot stalls —
sysinit.target is held back forever, SSH/LightDM never start, and every
console writer blocks (the boot looks completely frozen).

Reproduced on hardware with the pristine RC image: the wizard forks
(Forked systemd-firstboot as 893), never exits, sysinit.target: starting held back, waiting for: systemd-firstboot.service, for 16+ minutes.

The same image completes its whole first boot under QEMU (wizard finishes
in milliseconds), so the hang is specific to the real console path on
hardware. CI never catches this because distro_testing/scripts/prepare-image.sh
marks first boot as done for QEMU.

Fix (no masking — the unit still runs)

Ship a drop-in that detaches the wizard's stdio from the console:

/etc/systemd/system/systemd-firstboot.service.d/headless.conf
[Service]
StandardInput=null
StandardOutput=null
StandardError=null
  • Prompts read EOF and the wizard exits immediately (exit 0, verified offline
    with the image's own systemd-firstboot against the mounted image).
  • The unit still runs: machine-id per-install regeneration, credential
    import, and first-boot-complete.target are all preserved — future
    upstream first-boot logic keeps working (nothing masked).
  • All four prompted settings are pre-initialized at build time (locale.conf /
    vconsole.conf KEYMAP / localtime fallbacks + passwordless-root credential),
    so the wizard has nothing to ask even with a keyboard attached.
  • BASE_FIRSTBOOT_NONINTERACTIVE=no (new config knob) restores console stdio
    for distros that want an interactive wizard.

Hardware verification (Pi 5 rig, EEPROM menu → SD boot)

Test Result
Pristine image, wizard enabled FROZEN — child forked, no exit, 16 min, SSH closed
Pristine image under QEMU First boot completes (wizard Finished, clean reboot)
Patched image (headless.conf drop-in) First boot reaches GUI: /welcome/ → HTTP 200, LightDM active, welcome screen on HDMI, graphical.target queued
machine-id Regenerated per-install (fresh id each flash), first-boot-complete marker lifecycle intact

Screenshots + serial logs: pirig debug/2026-09-08*, /tmp/final-test.log
(poll log: welcome=200 from the first boot, no freezes).

Notes

  • Root cause of the block itself (console deadlock in the trixie kernel on
    Pi 5) deserves an upstream report — this PR only makes CustomPiOS images
    headless-safe by default.
  • The previously-opened approach (mask via /dev/null symlink) was replaced
    by this drop-in; the mask commit is dropped from the branch.

On Raspberry Pi 5 (Debian trixie, kernel 6.12.47) the firstboot wizard
child blocks opening /dev/console: sysinit.target stalls, SSH/LightDM never
start, and every console writer blocks — the boot appears frozen (reproduced
on hardware; the same image completes first boot under QEMU).

Verified fix on Pi 5 hardware: ship a drop-in detaching the unit's stdio
from the console. Prompts then read EOF and the wizard exits immediately
(exit 0, verified offline), while the unit still runs — machine-id
per-install regeneration, credential import and first-boot-complete.target
are all preserved (no masking). BASE_FIRSTBOOT_NONINTERACTIVE=no restores
console stdio for distros that ship a display/keyboard wizard.

Evidence: hang reproduced with the pristine wizard (forked child, no SIGCHLD,
16min), fixed boot reaches graphical.target with the drop-in.
@guysoft guysoft changed the title fix: mask systemd-firstboot wizard for headless images fix: keep systemd-firstboot running headless (detach stdio from console) Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant