Skip to content

fix(hooks): include hook output in the error when a hook fails - #14088

Open
dennislapchenko wants to merge 1 commit into
docker:mainfrom
dennislapchenko:fix/hook-failure-output
Open

fix(hooks): include hook output in the error when a hook fails#14088
dennislapchenko wants to merge 1 commit into
docker:mainfrom
dennislapchenko:fix/hook-failure-output

Conversation

@dennislapchenko

Copy link
Copy Markdown

What I did

A failing service hook reports only its exit code:

dependency failed to start: db hook exited with status 1

What the hook printed is not in the error. Worse, for a hook that runs without a listener the output does not exist anywhere: runHook created the exec with AttachStdout: false, AttachStderr: false, so the daemon discarded both streams, and runWaitExec only polled ExecInspect for the exit code. Nothing to read afterwards, no file, no log.

That path is taken by restart.go, run.go, and by every tool that drives compose as a library through Start - s.start() passes a nil listener, so StartOptions.Attach never reaches hooks.

Now stdout and stderr are always attached and the tail of them is kept. On non-zero exit it goes into the error:

db hook exited with status 1: SQLSTATE[42S02]: Base table or view not found

How

  • ExecCreate always attaches both streams, and the attach path is the only path.
  • A small outputTail keeps the last 10 lines / 2 KiB. It drops the oldest content, caps a line that never ends, and never short-writes, so a chatty hook is neither blocked nor able to grow the buffer.
  • With a listener the lines still stream to it exactly as before, the buffer only tees them.
  • Success is unchanged: the tail is dropped, nothing extra is printed or returned.
  • runWaitExec is removed. It existed only for the unattached branch, which is the branch that threw the output away.

Notes for review

  • Secrets. Hook output can contain sensitive text and it now appears in an error message. That is why it is capped and attached to the failure only - a passing hook adds nothing. If you would rather have it behind a flag, tell me which one and I change it.
  • Cancellation. The old unattached branch returned nil when the context was done, so a cancelled hook counted as success. The attach path returns the read error instead. I think that is the better answer, but it is a behaviour change worth naming.
  • Cost. One attach per hook, output read and dropped except the tail. Hooks are already synchronous, so no timing change.

Tests

pkg/compose/hook_test.go:

  • TestRunHook_FailureIncludesOutput - with and without a listener; asserts both streams are attached and the stderr text reaches the error.
  • TestRunHook_SuccessKeepsOutputOut - a passing hook returns nil and its line still goes to the listener.
  • TestOutputTail - line cap, missing trailing newline, blank lines, and the byte cap with a 4 KiB write that must not short-write.

go build ./..., go vet ./pkg/..., go test ./pkg/compose/ and golangci-lint run ./pkg/compose/... are green.

Why I care

I run a GitOps daemon that deploys compose stacks and notifies about failures. A migration hook failed there, the notification said hook exited with status 1, and the reason (Table 'service.sites' doesn't exist) was nowhere to be found - I had to re-run the script by hand inside the container to see it. With this change the reason travels with the error.

@dennislapchenko
dennislapchenko requested review from a team as code owners August 18, 2026 10:46
A failing service hook returns only its exit code:

    dependency failed to start: db hook exited with status 1

The output that explains why is not there, and for a hook running without a
listener it does not exist anywhere: runHook created the exec with
AttachStdout/AttachStderr false, so the daemon discarded both streams and
runWaitExec only polled for the exit code. There was nothing left to look at
afterwards.

Now stdout and stderr are always attached, and the last lines of them are kept
in a bounded buffer. On non-zero exit that tail goes into the error:

    db hook exited with status 1: SQLSTATE[42S02]: Base table or view not found

A successful hook is unchanged - output goes to the listener as before, nothing
is added anywhere. With a listener the lines still stream out, the buffer just
tees them.

The buffer keeps the last 10 lines and 2 KiB, drops the oldest content and never
short-writes, so a chatty hook cannot block on it and cannot grow it.

runWaitExec is removed: it existed only for the unattached branch, and that
branch is what threw the output away.

Signed-off-by: Deniss Solnce <dennis.lapchenko@gmail.com>
@dennislapchenko
dennislapchenko force-pushed the fix/hook-failure-output branch from 9c97daf to 4d43acc Compare August 18, 2026 10:47
@glours

glours commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Hey @dennislapchenko
Thanks for your contribution.

The pain point is real, but I'd rather not fix it here in isolation. The root issue is broader: hooks have no consistent observability story, no structured logging, no streaming to library callers, and your own note on secrets is a good illustration of why a one-off patch creates as many questions as it answers.

I started a rework that covers this area. I'd prefer hook output visibility lands as part of that work rather than as a standalone fix we'd have to reconcile later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants