Skip to content

Add result mode for multi-job deployments - #501

Open
GrantBirki wants to merge 3 commits into
github:mainfrom
GrantBirki:result-mode
Open

Add result mode for multi-job deployments#501
GrantBirki wants to merge 3 commits into
github:mainfrom
GrantBirki:result-mode

Conversation

@GrantBirki

Copy link
Copy Markdown
Contributor

This pull request adds opt-in result mode so multi-job deployments and noops can report their results without the extra reporting YAML. Result mode defaults to false and keeps existing modes and manual skip_completing workflows supported. It requires trusted context from the same workflow run and attempt, with one finalizer per context; interrupted reporting can still need manual recovery. Live sandbox tests cover ordinary deployments, forks, failure paths, and reruns at 920f085; both sandboxes are restored.

Closes #455

Copilot AI balanced review requested due to automatic review settings August 25, 2026 04:14
@GrantBirki

Copy link
Copy Markdown
Contributor Author

Live testing

Tested GrantBirki/branch-deploy@920f08566ce18bae326eedc2493059aff223f902 in 33 live runs plus two partial reruns. All Branch Deploy invocations in these test workflows used that exact commit, with harmless workflows on a personal same-repository PR, a same-repository PR in the org fork, and a cross-repository fork PR.

The table records behavior before cleanup. Failed and cancelled runs are intentional tests of failure handling; a failed reporting step does not necessarily mean the deployment work failed.

Group Case and run Observed behavior
Ordinary Mode omitted, explicit false, explicit false in org fork Normal success reporting and post-action lock release; no completion context added.
Ordinary Failed deployment Failure reported by the normal post action; original non-sticky lock released.
Ordinary Noop Normal noop feedback; no deployment or status record.
Manual Skip completion without a result action No automatic completion. Deployment stayed in_progress and the original lock remained for the caller.
Multi-job Personal sandbox success, org-fork success Successful completion from a later job, separate final report, original non-sticky lock released.
Multi-job Failed worker deployment_result: failure; failure status and feedback, failed result step, original lock released.
Multi-job Skipped required worker deployment_result: skipped; failure status and failed result step, original lock released.
Multi-job Successful noop, failed noop Matching success/failure feedback without creating a deployment or status record.
Multi-job Reusable work and finalizer workflows Completed the original operation in the same run and attempt; original lock released.
Cancellation Actual workflow cancellation Worker cancelled; deployment_result: cancelled, failure deployment status and failed result step. Original non-sticky lock retained.
Locks Sticky lock Completion succeeded; sticky lock retained.
Locks Replacement lock Completion succeeded; the changed non-sticky replacement lock was not deleted.
Locks Missing lock Completion succeeded without recreating the removed lock.
Locks Sentinel setup, locking disabled Setup created a sticky sentinel, not a deployment. Completion with locking disabled left that sentinel unchanged.
Settings Inherited settings Original template, labels and environment URL used, with the result job's DEPLOY_MESSAGE. Malformed result-job booleans were ignored while inheritance was enabled.
Settings Replace-all settings result_inherit_settings: false used default template/label settings and the receiver URL mapping, with the result job's DEPLOY_MESSAGE.
Forks Cross-repository fork deployment Completed the original verified fork SHA after a real approval and the required test1 check.
SHA binding PR head moved during work Worker observed the new PR head; completion still used the original admitted deployment SHA.
Reruns Initial run, result-only rerun, work rerun Initial attempt succeeded. Result-only rerun rejected stale context without changing completion state. Work rerun skipped at the caller guard; GitHub reused the prior result job, not a second validator execution.
Rejection Malformed context, altered deployment SHA Rejected with invalid_result_context.
Rejection Wrong repository, wrong run, wrong attempt Rejected with invalid_result_context.
Rejection Malformed job results, missing job results Rejected with invalid_result_inputs.
Rejection HTTP result URL Rejected with invalid_result_inputs.
Reporting Restricted finalizer permissions Deployment succeeded: deployment_result stayed success and a success status was recorded. Reporting/cleanup failed with result_completion_failed; no final comment, original lock retained.

All eight rejection cases failed the result step with an empty deployment_result; the original comments, reactions, deployment statuses and locks were unchanged. Where result-mode reporting completed, the final report was a separate comment and the original deployment-started comment stayed unchanged. End-of-run API checks found no duplicate completion from the action's post step. More detail is in the final-SHA sandbox evidence.

Both sandboxes were restored (personal restoration, org-fork restoration). Their main file trees and snapshotted settings match the starting state. Test PRs were closed without merging; disposable branches, locks, labels and environment configurations were removed. Existing PRs and the original dev lock were preserved. Original deployment records and statuses remain, with one separately labeled inactive cleanup status added to each test deployment; those cleanup statuses are not the outcomes in the table.

Result context and job results still need trusted workflow jobs in the same run and attempt, with one finalizer per context. These tests do not provide an exactly-once guarantee or guarantee finalization after force cancellation, runner loss or API failure.

Earlier testing at 5bbc408 is historical context, not additional passing evidence for the final SHA.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds opt-in result mode for safely finalizing multi-job deployments and noops while preserving existing completion behavior.

Changes:

  • Adds validated cross-job completion context and result aggregation.
  • Integrates final status, comment, label, reaction, and lock handling.
  • Documents the workflow and adds comprehensive tests.

Reviewed changes

Copilot reviewed 31 out of 33 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/types.ts Adds result-mode domain types.
src/operation-result.ts Registers result reason codes.
src/main.ts Dispatches result operations.
src/functions/trusted-deployment-template.ts Exports path validation.
src/functions/result-operation.ts Verifies and completes results.
src/functions/result-context.ts Parses and validates handoff data.
src/functions/prechecks.ts Captures deferred review data.
src/functions/post.ts Suppresses result-mode post handling.
src/functions/post-deploy.ts Adds result-aware completion and cleanup.
src/functions/post-deploy-message.ts Renders result statuses and URLs.
src/functions/environment-targets.ts Exports URL lookup.
src/functions/deployment-operation.ts Publishes trusted completion context.
src/functions/deferred-completion.ts Builds deferred handoff metadata.
src/action-io.ts Registers new inputs and outputs.
README.md Documents result-mode contracts.
docs/result-mode.md Adds the result-mode guide.
docs/examples.md Replaces manual reporting example.
action.yml Declares the public interface.
__tests__/types.test.ts Updates registry type checks.
__tests__/schemas/action.schema.yml Extends metadata schema.
__tests__/result-mode-fixtures.ts Adds shared context fixtures.
__tests__/main.test.ts Tests dispatch and context publication.
__tests__/functions/result-operation.test.ts Tests verification and completion.
__tests__/functions/result-context.test.ts Tests parsing and validation.
__tests__/functions/prechecks.test.ts Tests deferred review snapshots.
__tests__/functions/post.test.ts Tests post-hook suppression.
__tests__/functions/post-deploy.test.ts Tests result cleanup behavior.
__tests__/functions/post-deploy-message.test.ts Tests result message rendering.
__tests__/functions/deferred-completion.test.ts Tests context generation.
__tests__/docs-security.test.ts Enforces secure examples.
__tests__/action-contract.test.ts Updates public contract assertions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread README.md
Comment thread action.yml Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support result mode for multiple jobs usage

2 participants