Skip to content

Fleet Ops: manage the schema as Terraform, with CI for a policy repository - #11

Open
Zivxx wants to merge 4 commits into
fleet-ops-demofrom
fleet-ops-policy-as-code
Open

Zivxx wants to merge 4 commits into
fleet-ops-demofrom
fleet-ops-policy-as-code

Conversation

@Zivxx

@Zivxx Zivxx commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Stacked on #10. Adds the policy-as-code half of the Fleet Ops example: the same schema make setup writes through the API, as Terraform, plus the CI a policy repository needs to plan and apply it per environment.

What's in it

  • fleet-ops-demo/gitops/ is laid out like the repository Permit's GitOps reads, so it can be copied as is:
    • terraform/: the schema with the permitio/permit-io provider (0.0.26). It holds the same objects as internal/schema/schema.go: resource types, roles, relations, derivations, the two resource sets and their rules. cloud.tf keeps the state in HCP Terraform; delete it to keep the state locally.
    • .github/workflows/terraform.yaml:
      • on a pull request: format, validate, plan, and the plan posted as a comment;
      • on a push: main applies to Development and release/* to Production.
    • .github/workflows/policy-tests.yaml: opa check and opa test for the custom rules.
    • ci/permit_snapshot.py (standard library only): Permit deletes a role's assignments when the role is deleted, and a relation's tuples when the relation is deleted. Before each apply, this script saves what the plan deletes. When a revert is applied, it puts that data back. The README says plainly that this is the example's own code, not a Permit or Terraform feature, and lists its limits.
  • These workflows are templates. GitHub only runs workflows at the repository root, so nothing new runs in this repo.
  • This example's own CI gets a terraform job: terraform fmt -check, plus init -backend=false and validate, which need no credentials. It also compiles the snapshot script.
  • Also added: a terraform fmt pre-commit hook, a Dependabot entry for the provider, and a "Policy as code" paragraph and layout row in the example's README.

Review

The change went through the pr-pipeline. Both Python verifiers (idioms, risk) reviewed the snapshot script and the workflow that drives it, then pr-fixer adjudicated all 19 findings. Every finding was fixed, documented in gitops/README.md, or declined with a reason. The main ones:

  • Blocker, fixed: the restore step looked a snapshot up by artifact name. A fork's pull request could upload an artifact under that name, and it would be unzipped over the checkout and run with the production key in the environment. Now:

    • only this workflow's push runs on the branch are trusted;
    • the snapshot comes from the run that applied the reverted commit;
    • it's downloaded into $RUNNER_TEMP and read as data only.
  • Fixed:

    • retries on 5xx responses and network errors, honouring Retry-After;
    • bulk posts sent at most 1000 at a time, falling back to one item at a time when Permit refuses a batch;
    • pipefail, so a failed plan can't post an empty one;
    • concurrency grouped per environment, not per branch;
    • the log shows counts, never user keys;
    • save stops the apply when the plan deletes a resource type that still has instances, since no snapshot can bring those back.
  • Documented, not built:

    • reverting a commit from the middle of a push;
    • snapshots expiring after 30 days;
    • squash-merge settings that drop the "This reverts commit" line;
    • restore giving back access that was removed on purpose since the snapshot;
    • using one release branch at a time.

    Each of these makes the apply fail with an explanation, and the README says how to restore by hand.

One follow-up I made after the fixer (fa878a1): the revert search now starts from the last successful apply. Before, a revert whose run failed before applying would never be restored.

Tests

  • terraform fmt -check and validate (Terraform 1.13.5). The schema files are byte-identical to the baseline that was checked object by object against an environment created by make setup; schema.go has changed since only in comments.
  • actionlint with shellcheck, and zizmor, on all three workflows; pre-commit (gitleaks, typos, zizmor); py_compile.
  • The snapshot script against a local mock of Permit's facts API, built from reading the backend: matching by role and type, paging, replacements, chunking past 1000, refusals, an unknown environment, retries and timeouts. Permit's real API was not called.
  • The restore step, extracted verbatim from terraform.yaml, run through 14 scenarios with real git history and a gh stub: fork artifacts, a script inside an artifact, mid-push reverts, cancelled and failed runs, expired and missing snapshots, re-run attempts, rewritten history. Nothing ran on GitHub itself. The gh calls were checked against the stub and the docs only.

Names

The only organizations named are the example's made-up ones: Fleet Ops, Acme Freight, Lone Star Logistics, Summit Freight Group, Mesa Fleet Services and Freightline.

🤖 Generated with Claude Code

Zivxx and others added 4 commits October 19, 2026 14:56
…olicy repository

gitops/ is laid out like the repository Permit's GitOps reads: the schema in terraform/ (the same
objects internal/schema writes through the API), a workflow that plans each change on its pull
request and applies it on push (main to Development, release/* to Production), and
ci/permit_snapshot.py, which saves the role assignments and tuples a plan deletes and puts them back
when the change is reverted. The workflows are templates: GitHub only runs the ones at the
repository root.

The example's own CI checks the Terraform's format and validates it without credentials.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ounts instead of people

permit_snapshot.py, from the two verifier reports on 2024c09:

- Every call retries 429, 502, 503, 504, timeouts and refused connections with backoff, honouring
  Retry-After, and every failure ends as one line naming the call; a non-UTF-8 error body no longer
  raises. The retry count is one constant.
- restore posts at most 1000 operations per bulk call, the limit Permit's bulk routes enforce. When
  Permit rejects a call, it posts that call's items one at a time, so one assignment in a deleted
  tenant no longer blocks the rest. It reports the refusals; a refusal other than 404 or 409 fails it.
- restore checks which tuples already exist, as it already did for role assignments, so its counts
  are what it created. These listings run first and take a 404 as an error, so a wrong project or
  environment no longer reads as "everyone is gone". It refuses a snapshot saved in another
  environment.
- save stops the apply when the plan deletes or replaces a resource type that still has instances:
  Permit deletes the instances and everything on them, and no snapshot brings them back.
- save always writes a snapshot, an empty one when nothing is deleted, so a revert can tell
  "nothing to restore" from "snapshot missing".
- The logs show counts per role and per kind of relationship, never user keys; a refusal's text has
  the user key taken out.
- Configuration is read in main(), with an empty variable treated as unset, so the script imports
  cleanly and a missing setting names itself. PERMIT_API_URL must be http(s). Files are read and
  written through pathlib with an explicit encoding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…plied it

terraform.yaml and gitops/README.md, from the two verifier reports on 2024c09:

- The restore step no longer looks artifacts up by name across the repository, where a pull request
  from a fork can upload one under any name, and no longer unzips one over the checkout before running
  the checkout's script. It lists this workflow's push runs on this branch, takes the run that ended
  on the reverted commit, and downloads that run's snapshot into the runner's temporary folder, where
  only its JSON is read.
- Reverts are found with git log since the branch's last run that wasn't cancelled, not in the push
  event's commits, so a revert whose run GitHub cancelled while it waited is still restored. Only
  commits that changed terraform/ count.
- A revert whose snapshot can't be found, has expired, or was never kept fails the job with an error
  pointing at the README, instead of printing "that change deleted nothing".
- Artifact names carry the run attempt, so a re-run keeps its own snapshot.
- Both jobs use bash with pipefail, so a failed `terraform plan | tee` fails the step.
- The concurrency group is the target environment, so two release branches never apply to
  Production at once.
- The heredoc Python is gone: no Python left outside the script that the example's CI compiles.
- The README says what the step can't do and how to restore by hand: revert the commit a push ended
  on, keep git's revert line, 30 days of snapshots, failed runs, restored access and tenant
  membership, user keys in artifacts, one release branch at a time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… not the last run

A run that failed before applying (an HCP Terraform outage, say) moved the starting point past its own
revert, so the next push never restored it. Starting from the last successful run looks at a failed
run's reverts again; restoring twice is safe, since restore skips what's already there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant