Skip to content

fix: harden release builds and helmfile hydration - #48

Merged
panicboat merged 5 commits into
mainfrom
fix/release-and-hydrate-reliability
Oct 31, 2026
Merged

panicboat merged 5 commits into
mainfrom
fix/release-and-hydrate-reliability

Conversation

@panicboat

Copy link
Copy Markdown
Owner

What This PR Does

release の build と helmfile の hydrate で、黙って誤った結果になる 4 つの問題を直します。

What Needs to Change

  • component の入力がないと、空の manifest を書いて成功する。
    • Reason: Flux の prune: true で、実際のリソースが消えるおそれがある。
  • component を消しても、manifest が残る。
    • Reason: target が 0 件で hydrate の job が走らず、Flux が古い manifest を配り続ける。
  • 手動の release build が、タグではなく起動した ref のコードを build する。
    • Reason: tag=monolith-v0.9.0 で実行すると、main のコードで :v0.9.0 を上書きする。
  • タグの規則が場所によって違う。
    • Reason: route53 のような数字入りの service は、deploy はできても build ができない。

What Changes for Users

  • 入力のない component の hydrate は失敗します。
    • Before: 0 byte の manifest を書いて exit 0。
    • After: 対象のディレクトリを示して exit 1。
  • component を消した PR で、その manifest も消えます。
    • Before: kubernetes/manifests/<env>/<component> が残る。
    • After: hydrate の job が走り、manifest と index の行を消してコミットする。
  • 手動の release build は、タグのコードを build します。
    • Before: 起動した ref のコードを build し、:latest も動く。
    • After: タグのコードを build し、:latest は動かさない。
  • タグの service 名は [a-zA-Z0-9_-]+ になります。
    • Before: build は [a-z-]+、plan と deploy は [a-z0-9-]+。
    • After: すべて release-service.sh の同じ規則を使う。
flowchart LR
  Resolve["resolve"] --> Group["helmfile-targets-group"]
  Group -->|"target のある環境 + 孤児のある環境"| Hydrate["deploy-helmfile-hydrate"]
  Hydrate --> Index["hydrate-index.sh が孤児を消す"]
Loading

What to Review Closely

  • helmfile-targets-group が、helmfile の target がなくても毎回動くようになること。
    • Reason: 削除された component を見つけるには、target の有無によらず tree を見る必要があるため。
    • Details: What Was Decided

What Was Decided

  • 孤児の検出は、hydrate-index.sh と同じ定義で行う。
    • Reason: 2 つの判定がずれると、検出したのに消えない、または消えるのに検出しない、となるため。
    • 間違っていた場合の影響: 片方を変えるときにもう片方も直す必要がある。
  • render-diff.sh は、片側に component がないときに空の manifest を自分で書く。
    • Reason: 追加や削除の diff は hydrate-component.sh の 0 byte の出力に頼っていて、新しい確認で動かなくなるため。
    • 間違っていた場合の影響: 追加と削除の diff の表示が変わる。
  • :latest だけを手動の再 build から外し、sha と actor のタグは変えない。
    • Reason: monorepo の base manifest が参照しているのは :latest で、release のイベントの sha のタグは残したいため。
    • 間違っていた場合の影響: 手動の再 build で、main の sha のタグが古いコードの image を指す。

What Else Is Affected

Documents

  • 利用規約
  • プライバシーポリシー
  • ドキュメント

Systems & Operations

  • 呼び出し側には、Renovate が pin を上げたときに入ります。
  • 次のことは、実際の GitHub Actions の run で初めて確かめられます。
    • if と fromJson の matrix の評価順、auto-commit が削除もコミットすること、:latest の enable の評価、手動の build がタグのソースを使うこと。

How It Was Verified

  • for f in scripts/*/test.sh scripts/*/*.test.sh scripts/kubernetes-hydrate/tests/*.test.sh; do bash "$f"; done → すべて PASS
  • 新しいテストは、実装の前に落ちることを確かめました。
  • platform の production の component 22 件すべてに、helmfile.yaml か kustomization/ があることを確かめました。
  • platform の tree で beyla/production を消すと、orphan-environments.sh が ["production"] を返し、hydrate-index.sh が manifest と index の行を消すことを確かめました。
  • actionlint の増分は、既知の job.workflow_repository と job.workflow_sha の指摘だけでした。

🤖 Generated with Claude Code

https://claude.ai/code/session_01N1XYRqYewzhQfyJcM9zFjG

A component directory with neither helmfile.yaml nor kustomization/ wrote a
0-byte manifest and exited 0.
Flux with prune: true could then delete the live resources.
The check runs before the manifest is truncated, so the committed manifest stays.

render-diff.sh relied on that empty output for the side of a diff where a
component does not exist yet or anymore, so it now writes the empty side itself.
A directory that exists without an input reports a render failure there, as the
hydrator would.

Every kubernetes/components/*/production in platform's origin/main has one of
the two inputs, so no current component starts failing.

Claude-Session: https://claude.ai/code/session_01N1XYRqYewzhQfyJcM9zFjG
Signed-off-by: panicboat <panicboat@gmail.com>
Deleting kubernetes/components/<c>/<env>/ labels the pull request deploy:<c>,
but the resolver returns no helmfile target, so the hydrate job was skipped.
kubernetes/manifests/<env>/<c>/ stayed and Flux kept deploying it.

helmfile-targets-group now also lists each environment with an orphan directory,
with no services, so the hydrator runs only hydrate-index.sh for it.
The job needs both checkouts and runs whenever resolve succeeded, including when
no pull request was found and the targets output is empty.
deploy-helmfile-hydrate skips an empty list because fromJson('[]') fails the matrix.
deploy-helmfile and deployment-summary keep their conditions.

The detection reads the default merge ref like the resolve job, so it adds no
failure mode for a branch that no longer exists.

Claude-Session: https://claude.ai/code/session_01N1XYRqYewzhQfyJcM9zFjG
Signed-off-by: panicboat <panicboat@gmail.com>
reusable--release--build.yaml read the config at the tag, but the image builder's
checkout had no ref.
A manual workflow_dispatch with tag=monolith-v0.9.0 therefore built main's code and
overwrote :v0.9.0.

The image builder takes an optional ref, whose empty default keeps the label flow's
checkout, and the release build passes the tag.

Claude-Session: https://claude.ai/code/session_01N1XYRqYewzhQfyJcM9zFjG
Signed-off-by: panicboat <panicboat@gmail.com>
The release build parsed tags with [a-z-]+ and release-service.sh used [a-z0-9-]+,
while a service name is [a-zA-Z0-9_-]+ in deploy-actions.
route53 and oauth2-proxy could deploy but not build.

release-service.sh now holds the one rule for both modes, and the release build calls
it, which needs the shared workflows checkout that the plan and deploy workflows use.
The version is what follows "<service>-" in the validated tag, so it needs no second
rule, and greedy matching keeps the hyphens of a name such as github-oidc-auth.
The ::error:: for a bad tag now comes from the script.

The test that expected no checkout in detect-component now expects only the shared
workflows checkout, which keeps its intent that the consumer tree is not checked out.

Claude-Session: https://claude.ai/code/session_01N1XYRqYewzhQfyJcM9zFjG
Signed-off-by: panicboat <panicboat@gmail.com>
A release build now checks out its tag, so a manual workflow_dispatch of an old tag
from main would have pushed the old code as :latest.

The latest tag now also requires an empty ref.
A release event passes a ref as well, and is_default_branch is already false there
because its ref is the tag, so its tags do not change.
The sha and actor tags stay, because the release event's sha tag names the commit it
builds.

Claude-Session: https://claude.ai/code/session_01N1XYRqYewzhQfyJcM9zFjG
Signed-off-by: panicboat <panicboat@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cb2ae780-f58c-4062-a289-58e8ae32f279
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@panicboat
panicboat marked this pull request as ready for review October 31, 2026 00:26
@panicboat
panicboat merged commit 5181a1c into main Oct 31, 2026
7 checks passed
@panicboat
panicboat deleted the fix/release-and-hydrate-reliability branch October 31, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant