ci: the search index reaches master through a pull request opened with FRAMERS_BUMP_TOKEN, since master is protected

.github/WORKFLOWS.md

@@ -51,15 +51,23 @@ The Codex repository uses several GitHub Actions workflows for automated content
 - Cache restoration for faster builds
 - Parallel job execution where possible
 
-### 2. `auto-index.yml`
+### 2. `auto-index.yml` (Build and Publish Codex Index)
 
 **Triggers:**
-- Push to `weaves/`, `scripts/`, `schema/`
+- Push to any branch that changes `weaves/`, `scripts/`, `schema/` or this workflow
 - Manual dispatch
 - Repository dispatch (frame-deploy-complete)
 
 **Purpose:**
-Builds and publishes the search index JSON files.
+Builds the search index files, `codex-index.json`, `codex-search.json`, `codex-report.json` and `codex-blocks.json`, and on `master` publishes them through one pull request:
+- The build-index job builds the four files and archives them; on any other branch the run stops there
+- On `master`, the publish-artifacts job checks out `master`'s head and opens or updates one pull request into `master` from `ci/codex-index` carrying them, so the pull request carries the index alone and merges without a conflict even after `master` took an earlier index pull request. It never pushes to `master`, which is protected (below): a commit made in a job has none of the required checks
+- Pushes the branch and opens the pull request with `FRAMERS_BUMP_TOKEN`, as the weekly bump does, so the pull request gets Test Suite and rename-guard. Its commit message carries no skip marker: GitHub starts none of the `pull_request` workflows of a head commit whose message says `[skip ci]`, and the required checks would never report
+- Stops at its first step, with an error, when `FRAMERS_BUMP_TOKEN` is not available; both jobs' own `GITHUB_TOKEN` only reads, and the publish job installs and runs none of the repository's code
+- Publishes one run at a time, and keeps every waiting run (`queue: max`): GitHub's default keeps one, and a run that begins waiting replaces it whichever commit is newer. Each run waits for its own build, so a run of an earlier commit can reach the publish job after a later commit's run; when `master`'s head differs from the run's commit under `weaves/`, `scripts/`, `schema/` or this workflow, a later push changed them and its run publishes, so the earlier run leaves the pull request alone. A comparison that cannot be made fails the step, and nothing is published
+- A later run replaces the branch with its own build, and closes the pull request when the index no longer differs from `master`'s
+- Labels nothing: `landing-follows-the-bump-checks.yml` adds `landing` once the pull request's Test Suite and rename-guard pass on its head, and the pull request lands after its review comments are read, answered and resolved
+- `tests/index-pull-request.test.js` reads the workflow, runs its first step, and runs its ordering step in scratch repositories, since GitHub runs the publish job only on `master`
 
 ### 3. `ai-enhance-pr.yml`
 
@@ -163,25 +171,29 @@ Moves every `@framers/*` pin in a `package.json` to the newest version on npm, r
 - Labels nothing: `landing-follows-the-bump-checks.yml` adds `landing` once the pull request's Test Suite and rename-guard pass on its head, and the pull request lands after its review comments are read, answered and resolved
 - `tests/bump-framers-deps.test.js` reads the workflow, runs its first step and its npm lockfile refresh, and makes its commit in a scratch repository holding this repository's files, since GitHub reports a broken file only when the schedule or a dispatch starts it
 
-### 10. `landing-follows-the-bump-checks.yml` (Landing follows the bump checks)
+### 10. `landing-follows-the-bump-checks.yml` (Landing follows the bump and index checks)
 
 **Triggers:**
-- A Test Suite or rename-guard run requested, started (`in_progress`) or completed on `chore/bump-framers-deps`; a re-run sends no requested event, only the other two
+- A Test Suite or rename-guard run requested, started (`in_progress`) or completed on `chore/bump-framers-deps` or `ci/codex-index`; a re-run sends no requested event, only the other two
 
 **Purpose:**
-Keeps the `landing` label on the weekly bump's pull request only on a head whose guards passed; its guards are that pull request's own Test Suite and rename-guard runs. Whichever event starts it, it reads each guard's newest pull request run on the pull request's head (listed by the guard's workflow file):
+Keeps the `landing` label on the weekly bump's pull request and on the index's only on a head whose guards passed; a pull request's guards are its own Test Suite and rename-guard runs. It takes the branch from the run, and a run from any other branch changes nothing. Whichever event starts it, it reads each guard's newest pull request run on that branch's pull request's head (listed by the guard's workflow file):
 - Both passed: adds `landing` (created on first use) and keeps the guards comment (github-actions[bot]'s comment that starts with `<!-- landing-guards -->`) naming the two runs; when that comment cannot be written, the label comes off again
 - Either ended without passing: takes the label off and says which in the guards comment
 - Either has no run yet or has not finished, as after a push, a reopen or a re-run: keeps the label off and says so in a guards comment already there
 - Then reads the pull request and the runs again and writes again while the head or the verdict changed, so an event handled late, a guard finishing meanwhile or a push keeps the last word with the latest state; a pull request that left `master` while it was labelled loses the label
 - A run for an older head changes nothing; a change of base is otherwise left to `landing-follows-the-base.yml`
 - No concurrency group: GitHub cancels a group's waiting run when a newer one queues, and a cancelled run would show on `master`'s last commit
 - `workflow_run` runs the file as it reads on `master` with a token that can write, so the job checks out no code, runs only for a pull request from this repository's own branch (a fork's branch of the same name is skipped) and reads the run and the pull request inside its script; it never merges
-- `tests/bump-framers-deps.test.js` runs its script against a stand-in for the GitHub API, since GitHub runs the file only once it is on `master`
+- `tests/bump-framers-deps.test.js` runs its script against a stand-in for the GitHub API, for both branches, since GitHub runs the file only once it is on `master`
 
 ### The `landing` label
 
-No workflow here merges a pull request or turns on GitHub's auto-merge. A pull request is merged only after every review and comment on it has been read, fixed or answered and its threads resolved. Where a workflow's checks and guards pass, it adds the `landing` label (created on first use) and posts one comment saying which guards passed; when that comment cannot be posted, the label comes off again. A change of base takes the label off too, since the guards ran against the old base. On the weekly bump's pull request the guards are its own Test Suite and rename-guard runs: the label goes on once both pass on its head, and comes off when a push, a reopen or a re-run starts either again there or when either fails. A maintainer lands the pull request after that pass.
+No workflow here merges a pull request or turns on GitHub's auto-merge. A pull request is merged only after every review and comment on it has been read, fixed or answered and its threads resolved. Where a workflow's checks and guards pass, it adds the `landing` label (created on first use) and posts one comment saying which guards passed; when that comment cannot be posted, the label comes off again. A change of base takes the label off too, since the guards ran against the old base. On the weekly bump's pull request and on the index's, the guards are its own Test Suite and rename-guard runs: the label goes on once both pass on its head, and comes off when a push, a reopen or a re-run starts either again there or when either fails. A maintainer lands the pull request after that pass.
+
+### `master`'s protection
+
+`master` is protected. A change reaches it only once Unit Tests, Vocabulary & NLP Tests, Cache Tests and Integration Tests (Test Suite) and guard (rename-guard) have passed on it, each accepted from GitHub Actions only; a pull request need not be up to date with `master` first. The rule binds administrators too, and force-pushes and deletion are blocked. Those five run on every pull request into `master`. A commit made in a job has none of them, so `master` refuses a job's push: Update Badges writes the `badges` branch, and the index and the weekly bump arrive as pull requests. Update Badges runs on `master` only and is not required.
 
 ## Caching Strategy
 
@@ -216,7 +228,7 @@ restore-keys:
 | `ANTHROPIC_API_KEY` | AI only | Claude API key for AI enhancement |
 | `OPENAI_API_KEY` | AI only | OpenAI API key (fallback) |
 | `GH_PAT` | Yes | Personal access token for cross-repo triggers |
-| `FRAMERS_BUMP_TOKEN` | Bump only | framerslab organization secret shared with this repository: the personal token the weekly bump pushes its branch and opens its pull request with, so the pull request's checks run |
+| `FRAMERS_BUMP_TOKEN` | Bump and index | framerslab organization secret shared with this repository: the personal token the weekly bump and the index publish push their branches and open their pull requests with, so the pull requests' checks run |
 
 ## Manual Triggers
 

.github/workflows/auto-index.yml

@@ -1,5 +1,19 @@
 name: Build and Publish Codex Index
 
+# The build-index job builds codex-index.json, codex-search.json, codex-report.json and codex-blocks.json and archives
+# them. On master, publish-artifacts then opens or updates one pull request into master from ci/codex-index carrying
+# them. It never pushes to master: master is protected, and a commit reaches it only once its required checks have
+# passed, which a commit made in a job has not ("About protected branches", Require status checks before merging:
+# https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches#require-status-checks-before-merging).
+#
+# The branch is pushed and the pull request opened with FRAMERS_BUMP_TOKEN, a personal token held as a framerslab
+# organization secret, as bump-framers-deps.yml does: a pull request a workflow opens with GITHUB_TOKEN gets its
+# pull_request runs in an approval-required state, with no job, so its required checks would never report. The commit
+# message carries no skip marker, since GitHub starts none of the pull_request workflows of a head commit whose message
+# says [skip ci]. Nothing here labels or merges the pull request: landing-follows-the-bump-checks.yml adds the landing
+# label once its Test Suite and rename-guard pass on its head, and it lands after its review comments are read, answered
+# and resolved. tests/index-pull-request.test.js reads this file.
+
 on:
   push:
     paths:
@@ -12,6 +26,10 @@ on:
     types:
       - frame-deploy-complete
 
+# The pull request goes through FRAMERS_BUMP_TOKEN, so each job's own GITHUB_TOKEN only reads.
+permissions:
+  contents: read
+
 jobs:
   build-index:
     runs-on: ubuntu-latest
@@ -47,7 +65,7 @@ jobs:
         run: pnpm run build:search
 
       - name: Upload artifacts
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v7
         with:
           name: codex-search-assets
           path: |
@@ -59,23 +77,89 @@ jobs:
   publish-artifacts:
     needs: build-index
     runs-on: ubuntu-latest
-    if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main'
+    if: github.ref == 'refs/heads/master'
+    # One run at a time, so two runs never push the pull request branch at once. GitHub keeps one waiting job by default,
+    # and a job that begins waiting replaces it whichever commit is newer, so an older commit's build that finished later
+    # would cancel the newer commit's publish. queue: max keeps every waiting job, in the order each began to wait, and
+    # each decides when its turn comes (the ordering step below), so the run of the newest change publishes
+    # ("Control the concurrency of workflows and jobs":
+    # https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency).
+    concurrency:
+      group: codex-index-pull-request
+      cancel-in-progress: false
+      queue: max
+    # FRAMERS_BUMP_TOKEN is a personal token that reaches past this repository, and a step can change what a later step of
+    # its job runs, so this job installs and runs none of the repository's code, and its third-party action is pinned to a
+    # commit, with the release that commit carries named beside it.
     steps:
+      # First: without the secret, create-pull-request stops with "Input 'token' not supplied" only at its own step.
+      - name: FRAMERS_BUMP_TOKEN is available
+        env:
+          FRAMERS_BUMP_TOKEN: ${{ secrets.FRAMERS_BUMP_TOKEN }}
+        run: |
+          if [ -z "$FRAMERS_BUMP_TOKEN" ]; then
+            echo "::error::FRAMERS_BUMP_TOKEN is not available here; a pull request opened with GITHUB_TOKEN gets no CI job."
+            exit 1
+          fi
+
+      # master's head, whichever commit this run built: create-pull-request starts the pull request from the branch checked
+      # out, so the pull request carries the index alone and merges without a conflict even after master took an earlier
+      # index pull request.
       - uses: actions/checkout@v5
-
-      - uses: actions/download-artifact@v4
+        with:
+          ref: master
+
+      # Each run waits for its own build, so a run of an earlier commit can get here after a later commit's run. The index
+      # is built from the paths whose change starts this workflow (on.push.paths). When master's head differs from this
+      # run's commit under them, a later push changed them and its run publishes, so this run leaves the pull request alone
+      # instead of putting an older index on it. When they match, no later run publishes, so this one does. A comparison
+      # that cannot be made fails the step, and nothing is published.
+      - name: Leave the index to the run of a later change
+        id: order
+        run: |
+          git fetch --quiet --depth=1 origin "$GITHUB_SHA"
+          status=0
+          git diff --quiet "$GITHUB_SHA" HEAD -- weaves scripts schema .github/workflows/auto-index.yml || status=$?
+          case "$status" in
+            0)
+              echo "later=false" >> "$GITHUB_OUTPUT"
+              ;;
+            1)
+              echo "::notice::master changed weaves/, scripts/, schema/ or this workflow after ${GITHUB_SHA::7}; the run of that change publishes the index."
+              echo "later=true" >> "$GITHUB_OUTPUT"
+              ;;
+            *)
+              echo "::error::git diff could not compare ${GITHUB_SHA::7} with master's head (exit $status)."
+              exit "$status"
+              ;;
+          esac
+
+      - uses: actions/download-artifact@v8
+        if: steps.order.outputs.later == 'false'
         with:
           name: codex-search-assets
           path: .
 
-      - name: Commit and push JSON assets
-        run: |
-          git config user.name "github-actions[bot]"
-          git config user.email "github-actions[bot]@users.noreply.github.com"
-          git add codex-index.json codex-search.json codex-report.json codex-blocks.json
-          if ! git diff --cached --quiet; then
-            git commit -m "ci(codex): update search index and block tags [skip ci]"
-            git push
-          else
-            echo "No changes to commit."
-          fi
+      - name: Open or update the index pull request
+        if: steps.order.outputs.later == 'false'
+        uses: peter-evans/create-pull-request@22a9089034f40e5a961c8808d113e2c98fb63676 # v7.0.11
+        with:
+          # the personal token, so the pull request's own events start Test Suite and rename-guard (see the file's top);
+          # branch-token defaults to it, so the branch is pushed with it too
+          token: ${{ secrets.FRAMERS_BUMP_TOKEN }}
+          commit-message: 'ci(codex): update search index and block tags'
+          title: 'ci(codex): update search index and block tags'
+          body: |
+            The search index files built from ${{ github.sha }} by run ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}: `codex-index.json`, `codex-search.json`, `codex-report.json` and `codex-blocks.json`.
+
+            master is protected, so the index reaches it through this pull request. The Build and Publish Codex Index workflow opens and updates it with `FRAMERS_BUMP_TOKEN`, so its Test Suite and rename-guard run; `landing-follows-the-bump-checks.yml` labels it `landing` once both pass on its head, and it lands after its review comments are read, answered and resolved. A later run replaces this branch with its own build, and closes the pull request when the index no longer differs from master's.
+          branch: ci/codex-index
+          base: master
+          delete-branch: true
+          # The four files the build job archives, each tracked here. The action stages them with one git add -- <add-paths>,
+          # which stages nothing when one path matches no file; tests/index-pull-request.test.js checks the list.
+          add-paths: |
+            codex-index.json
+            codex-search.json
+            codex-report.json
+            codex-blocks.json