From dd5780a1882fbf162c8b592a86b9ad6f9ecfc366 Mon Sep 17 00:00:00 2001 From: Derek Lewis Date: Tue, 22 Sep 2026 00:41:02 +0000 Subject: [PATCH] =?UTF-8?q?=F0=9F=8F=97=EF=B8=8F=E2=9C=A8=EF=BC=9Afetch=20?= =?UTF-8?q?the=20SDK's=20API=20reference=20ourselves?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The SDK builds its API reference on the run that publishes and attaches it as sdk-api-docs; this portal renders it from vendor/sdk-api/. Between the two was a person downloading a zip, and nothing failed when nobody did: the packages went to the registry either way and the reference stayed a release behind. The SDK API sync workflow makes that walk. Given the ID of the Release run that published, it downloads the artifact, places it here, imports it, and opens a pull request. The import is what decides: it validates the manifest, confines every path the manifest names, and refuses a corpus it could not render, so a refused artifact fails the run instead of becoming a pull request. This portal fetches rather than the SDK pushing. The SDK's release job holds the credential that publishes to the registry, and a second one that could write here would widen what a mistake in that job reaches. Everything needed to refuse a bad artifact is already here. Inside the job the same reasoning splits the credential in two, each as narrow as the one thing it does. The reading token is scoped to the SDK with `permission-actions: read`, which is what downloading one artifact needs; without it the token would inherit every permission the app was installed with. The checkout keeps no credentials, as every other workflow here does. The writing token is scoped to this repository alone and is not minted until the download, the install and the import have all finished, so nothing that runs anybody else's code runs while it exists — lifecycle scripts are a dependency's own code, and anything they can read they can send. The push is given that token through a credential helper rather than a remote URL, since a URL carrying a token is written into .git/config, printed by `git remote -v`, and copied into the output of anything that reports what it fetched. It opens the pull request with an app token rather than GITHUB_TOKEN, which raises one that starts no checks — the reason vendored-sync.yml files an issue instead. Here the check is the whole point. Placing the artifact is a task rather than a line of shell, because three things cannot be checked afterwards: that a release is added rather than replaced, since the reference a release shipped with is not this repository's to rewrite; that a directory is named for a release, since that name becomes a public URL; and that nothing arrived by symbolic link. Nothing makes the handover happen, so SDK API drift asks the SDK weekly what it has released and files an issue when a release is missing its reference here. It reads tags rather than the registry: every package in the SDK was published for years from a repository of its own, and those versions are served still, so the registry would report a reference missing for releases that never had one. Signed-off-by: Derek Lewis Assisted-by: Claude-Code:claude-opus-5 --- .github/workflows/sdk-api-drift.yml | 123 ++++++++++ .github/workflows/sdk-api-sync.yml | 198 ++++++++++++++++ build/shared/sdk-release.mts | 116 ++++++++++ build/shared/sdk-release.test.mts | 93 ++++++++ build/shared/sdk-unpack.test.mts | 292 ++++++++++++++++++++++++ build/tasks/check-sdk-api-drift.mts | 166 ++++++++++++++ build/tasks/unpack-sdk-api-artifact.mts | 207 +++++++++++++++++ package-scripts.yml | 9 + package.json | 1 + vendor/sdk-api/README.md | 23 +- 10 files changed, 1225 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/sdk-api-drift.yml create mode 100644 .github/workflows/sdk-api-sync.yml create mode 100644 build/shared/sdk-release.mts create mode 100644 build/shared/sdk-release.test.mts create mode 100644 build/shared/sdk-unpack.test.mts create mode 100644 build/tasks/check-sdk-api-drift.mts create mode 100644 build/tasks/unpack-sdk-api-artifact.mts diff --git a/.github/workflows/sdk-api-drift.yml b/.github/workflows/sdk-api-drift.yml new file mode 100644 index 000000000..bb9c1901a --- /dev/null +++ b/.github/workflows/sdk-api-drift.yml @@ -0,0 +1,123 @@ +# Watching for an SDK release whose API reference never arrived here. +# +# Handing a release's reference over is a step of its own -- the **SDK API +# sync** workflow beside this one, or the manual walk through "Publishing the +# API reference" in the SDK's `RELEASING.md`. Nothing fails when it is missed. +# The packages are on the registry either way, `/docs/sdk/` simply goes on +# listing the releases it already had, and the omission keeps until somebody +# notices the reference is a version behind. This is what notices. +# +# Weekly rather than on every pull request, because this reaches the network: +# an API that is slow, rate-limiting or unreachable would otherwise fail +# changes that have nothing to do with it. That is the same line +# `vendored-sync.yml` draws, and the reason `check-sdk-api-drift.mts` sits +# outside `verify/` with `check-vendored.mts`. +# +# It opens an issue rather than a pull request. What to do about a missing +# reference is to run the sync workflow with a run ID only a person can look +# up, so there is nothing here for a pull request to carry. +# +# Actions are pinned by commit, never by tag. +name: SDK API drift + +on: + schedule: + # Friday, clear of the other three scheduled runs. + - cron: '0 5 * * 5' + workflow_dispatch: + +permissions: + contents: read + issues: write + +# A scheduled run and a hand-started one should not both file the same report. +concurrency: + group: ${{ github.workflow }} + cancel-in-progress: false + +jobs: + check: + name: Check the published API reference + runs-on: ubuntu-latest + steps: + - name: Check out project repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + # Nothing here pushes, and a token left in .git/config is one more + # thing that could be picked up by something that should not have it. + persist-credentials: false + + - name: Set up Node.js runtime + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version-file: 'package.json' + + # Unlike `vendored-sync.yml`, this one installs. The task orders + # releases with the same comparison `/docs/sdk/` lists them by, which + # lives beside the HTML sanitizing in `build/shared/sdk-docs.mts`, so + # node has `sanitize-html` to resolve before any of it runs. Sharing + # that comparison is worth an install on a weekly job; writing a second + # one that sorted 10.0.0 below 4.2.0 would not be. + - name: Install + run: | + # No corepack: node is unbundling it, and pnpm reads + # `packageManager` and fetches that version itself. Installing the + # named one rather than the newest saves fetching pnpm twice. + npm install --global "pnpm@$(node -p "require('./package.json').packageManager.replace('pnpm@','').split('+')[0]")" + pnpm install + + - name: Compare against the SDK's releases + id: compare + env: + # The API rate-limits an anonymous caller by runner address, which + # is shared, so the tags are asked for as this repository. + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + node build/tasks/check-sdk-api-drift.mts > report.md || code=$? + cat report.md + + # Both bits are read: being behind says nothing about whether the + # tags could be read, so neither answer is allowed to hide the + # other. Reported after the issue is filed, so an API nobody could + # reach cannot hold back a report about a reference that is missing. + echo "behind=$(( (${code:-0} & 1) != 0 ))" >> "$GITHUB_OUTPUT" + echo "unchecked=$(( (${code:-0} & 2) != 0 ))" >> "$GITHUB_OUTPUT" + + - name: Say so, once + if: steps.compare.outputs.behind == '1' + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + TITLE: 📖 an SDK release is missing its API reference here + run: | + # Matched against the open issues themselves rather than through + # search, which is an index and lags behind what was just written. + # `gh` pages until it has as many as asked for, and the number is + # far past what this repository will hold, so the report cannot be + # missed and a duplicate filed beside it. + # One issue at a time: a weekly comment on a report nobody has acted + # on yet says nothing the report did not. + open=$(gh issue list --state open --limit 1000 --json number,title \ + --jq 'map(select(.title == env.TITLE)) | .[0].number // empty') + + if [ -n "$open" ]; then + echo "already reported in #${open}" + exit 0 + fi + + { + echo 'The SDK has published a release this portal does not' + echo 'document. Its reference is handed over separately from the' + echo 'packages, and nothing fails when that step is missed.' + echo + cat report.md + } > body.md + + gh issue create --title "$TITLE" --body-file body.md + + - name: Fail if the releases could not be read + # `always()`, so this runs after a report has been filed rather than + # instead of one. + if: always() && steps.compare.outputs.unchecked == '1' + run: | + echo '::error::the SDK'"'"'s releases could not be read' + exit 1 diff --git a/.github/workflows/sdk-api-sync.yml b/.github/workflows/sdk-api-sync.yml new file mode 100644 index 000000000..2802295d6 --- /dev/null +++ b/.github/workflows/sdk-api-sync.yml @@ -0,0 +1,198 @@ +# Bringing one SDK release's API reference into this repository. +# +# The SDK generates its reference as TypeDoc Markdown and attaches it to the +# run that published, as `sdk-api-docs`. This portal renders it from +# `vendor/sdk-api//`. Carrying it between the two was a person +# downloading a zip and unpacking it here, and nothing failed when nobody did: +# the packages went to the registry either way and the reference simply stayed +# a release behind. +# +# This portal pulls rather than the SDK pushing. The SDK's Release job holds +# the credential that publishes to the registry, and giving that same job a +# second one that could write here would widen what a mistake in it reaches. +# Everything needed to refuse a bad artifact is already in this repository, so +# the fetching belongs here as well. +# +# It opens a pull request rather than pushing to the default branch, and mints +# an app token to do it. A pull request raised with `GITHUB_TOKEN` does not +# start the checks -- which is why `vendored-sync.yml` files an issue instead +# -- and here the checks are the whole point: the import task refuses a corpus +# this portal could not render, so a green pull request is what says the +# reference will appear. An app token starts them. +# +# First-time setup. This needs a GitHub App of its own, installed on both +# repositories, with its ID in `SDK_SYNC_APP_ID` and its private key in +# `SDK_SYNC_APP_PRIVATE_KEY`: +# +# - on `sdk`: Actions: read (to download the run's artifact) +# - on `openinf.github.io`: Contents: write, Pull requests: write +# +# Not the app `commit-queue.yml` uses. That one can merge to the default +# branch, and the way to keep that narrow is to not hand it a second errand +# in another repository. +# +# Actions are pinned by commit, never by tag. +name: SDK API sync + +on: + workflow_dispatch: + inputs: + run-id: + description: 'The SDK Release run that published, by its run ID' + required: true + type: string + +permissions: + contents: read + +# Two releases handed over at once would write into the same directory and +# race each other onto the same branch. +concurrency: + group: ${{ github.workflow }} + cancel-in-progress: false + +jobs: + sync: + name: Import the release's API reference + runs-on: ubuntu-latest + steps: + # Two tokens, each as narrow as the one thing it does, because this job + # runs `pnpm install` in the middle of itself. Lifecycle scripts are a + # dependency's own code, and anything they can read they can send: a + # single credential that could write here, sitting in the environment + # or in .git/config while they run, is one an install away from being + # somebody else's. So the reading token cannot write, the writing token + # is not minted until every step that runs foreign code has finished, + # and neither is scoped to a repository it has no business in. + - name: Mint a token that can only read the SDK + id: read-token + uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 + with: + app-id: ${{ secrets.SDK_SYNC_APP_ID }} + private-key: ${{ secrets.SDK_SYNC_APP_PRIVATE_KEY }} + owner: ${{ github.repository_owner }} + repositories: sdk + # Without these it inherits every permission the app was installed + # with, which is more than downloading one artifact needs. + permission-actions: read + + - name: Check out project repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + # Left out of .git/config the way every other workflow here leaves + # it out. The push below brings its own, once nothing foreign is + # left to run. + persist-credentials: false + + - name: Set up Node.js runtime + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version-file: 'package.json' + + - name: Download the release's API reference + env: + GH_TOKEN: ${{ steps.read-token.outputs.token }} + # Read from the environment rather than interpolated into the + # command: an input is written by whoever started the run, and one + # pasted into a shell is one that can run as code. + RUN_ID: ${{ inputs.run-id }} + SDK: ${{ github.repository_owner }}/sdk + run: | + gh run download "$RUN_ID" --repo "$SDK" --name sdk-api-docs \ + --dir "$RUNNER_TEMP/sdk-api-docs" + + # No install step: the task reads only what node ships with, so there is + # nothing to resolve before it can refuse a bad artifact. What it + # enforces is what cannot be enforced afterwards -- that a release is + # added rather than replaced, that its directory is named for a release, + # and that nothing arrives by symbolic link. + - name: Place it under vendor/sdk-api/ + id: unpack + run: | + node build/tasks/unpack-sdk-api-artifact.mts \ + --from "$RUNNER_TEMP/sdk-api-docs" + + - name: Install + run: | + # No corepack: node is unbundling it, and pnpm reads + # `packageManager` and fetches that version itself. Installing the + # named one rather than the newest saves fetching pnpm twice. + npm install --global "pnpm@$(node -p "require('./package.json').packageManager.replace('pnpm@','').split('+')[0]")" + pnpm install + echo "$(pwd)/node_modules/.bin" >> $GITHUB_PATH + + # The real import, against the real artifact, before a pull request + # exists to review. It validates the manifest, confines every path the + # manifest names, and refuses a corpus it cannot map onto portal URLs. + # The same task runs again on the pull request; failing here means a + # refused artifact never becomes one. + - name: Check the portal can render it + run: nps compile.importSdkApiDocs + + # Only now, with the download unpacked, the install done and the import + # run, is a credential that can write minted. Nothing after this point + # runs anybody else's code. + - name: Mint a token that can only write here + id: write-token + uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 + with: + app-id: ${{ secrets.SDK_SYNC_APP_ID }} + private-key: ${{ secrets.SDK_SYNC_APP_PRIVATE_KEY }} + owner: ${{ github.repository_owner }} + repositories: openinf.github.io + permission-contents: write + permission-pull-requests: write + + - name: Open a pull request + env: + GH_TOKEN: ${{ steps.write-token.outputs.token }} + VERSIONS: ${{ steps.unpack.outputs.versions }} + RUN_ID: ${{ inputs.run-id }} + SDK: ${{ github.repository_owner }}/sdk + # What the app commits as. The `[bot]` suffix is reserved -- no + # account can be named with it -- and is what `verify.commits` reads + # to leave a machine's commit messages alone. + APP_SLUG: ${{ steps.write-token.outputs.app-slug }} + APP_ID: ${{ secrets.SDK_SYNC_APP_ID }} + run: | + branch="sdk-api/$(printf '%s' "$VERSIONS" | tr -d ' ' | tr ',' '-')" + + git config user.name "${APP_SLUG}[bot]" + git config user.email \ + "${APP_ID}+${APP_SLUG}[bot]@users.noreply.github.com" + + # The checkout kept no credentials, so the push is given one here. + # Through the credential helper rather than in the remote URL: a URL + # carrying a token is written into .git/config, printed by `git + # remote -v`, and copied into the output of anything that reports + # what it fetched. + git config credential.helper \ + '!f() { echo "username=x-access-token"; echo "password=$GH_TOKEN"; }; f' + + git switch -c "$branch" + git add vendor/sdk-api + git commit --file - </api/, and the release is listed on + /docs/sdk/. + + A release adds a version rather than replacing one, so a link into + an older reference keeps describing the release it was published + with. + COMMIT + + git push -u origin "$branch" + + gh pr create --title "📖✨:publish the SDK ${VERSIONS} API reference" \ + --body "Before: this portal documented no \`${VERSIONS}\` API reference, and a reader following a link to it found nothing. + + After: \`/docs/sdk/${VERSIONS}/api/\` renders the reference the SDK published, in the portal's own documentation layout, and \`/docs/sdk/\` lists the release. + + This is the generated TypeDoc Markdown from the SDK's Release run \`${RUN_ID}\`, which built it from the versions that shipped and the commit they were published from. + + How: the artifact was downloaded from that run and placed under \`vendor/sdk-api/${VERSIONS}/\` unchanged. Nothing here is edited by hand. The build imports it, validating the manifest and refusing any page it cannot map onto a portal URL, so this pull request's own check is what confirms the reference renders. + + Opened by the **SDK API sync** workflow." diff --git a/build/shared/sdk-release.mts b/build/shared/sdk-release.mts new file mode 100644 index 000000000..4bc65319c --- /dev/null +++ b/build/shared/sdk-release.mts @@ -0,0 +1,116 @@ +/** + * @file Reading SDK releases, and saying whether the portal is behind them. + * @author The OpenINF Authors & Friends + * @license MIT OR Apache-2.0 OR BlueOak-1.0.0 + * @module {type ES6Module} build/shared/sdk-release + * + * The deciding is here rather than in `check-sdk-api-drift.mts` so that it + * can be read without reaching the network. What the task keeps is the two + * things that do: asking the SDK repository for its tags, and reading the + * directories in this checkout. + */ + +import { compareVersions } from '@openinf/portal/build/sdk-docs'; + +/** A release version, as the portal, the SDK and Changesets all spell one. */ +const VERSION = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/; + +/** What this portal is being compared against, and where its releases sit. */ +const ARTIFACTS = 'vendor/sdk-api'; + +/** + * The release a tag names, if it names one. + * + * Changesets writes `@scope/name@1.2.3` for a scoped package, so the version + * is what follows the last `@` rather than the first -- the scope's own `@` + * is the one at the start. A tag that is not a release, such as someone's + * marker, is not an error; it is simply not an answer, and is left out. + * @param {string} tag The tag name as the API gives it. + * @returns {string | undefined} The version it names, if it names one. + */ +export function releaseOf(tag: string) { + const at = tag.lastIndexOf('@'); + const candidate = at <= 0 ? tag.replace(/^v/, '') : tag.slice(at + 1); + + return VERSION.test(candidate) ? candidate : undefined; +} + +/** + * The newest of a set of versions, by precedence rather than by name. + * Sorting by name would put 10.0.0 below 4.2.0, which is the same mistake the + * release listing on `/docs/sdk/` avoids. + * @param {string[]} versions The versions to choose between. + * @returns {string | undefined} The newest, or nothing when there are none. + */ +export function newestRelease(versions: string[]) { + return [...versions].sort(compareVersions).at(0); +} + +/** Whether the portal is behind, and what to say about it. */ +export type Drift = { behind: boolean; report: string }; + +/** + * Compares the newest release the SDK has tagged against the newest one this + * portal documents, and writes what a reader should do about the answer. + * + * A portal documenting nothing is only behind once a release exists to be + * behind. Before the SDK's first tag there is no artifact anywhere, so there + * is nothing missing here -- which is not what asking the registry would have + * said, since every package in the SDK shipped for years from a repository of + * its own and those versions are still served. + * @param {string | undefined} released The newest release the SDK has tagged. + * @param {string | undefined} documented The newest release published here. + * @returns {Drift} Whether to report, and the report. + */ +export function driftReport( + released: string | undefined, + documented: string | undefined +): Drift { + if (released === undefined) { + return { + behind: false, + report: + 'The SDK has tagged no release, so there is no API reference for ' + + 'this portal to be missing.', + }; + } + + if (documented === undefined) { + return { + behind: true, + report: [ + `The SDK has released ${released}, and this portal documents no`, + 'release at all.', + '', + `\`${ARTIFACTS}/\` holds no version directory, so \`/docs/sdk/\``, + 'renders its placeholder rather than an API reference.', + '', + 'Run the **SDK API sync** workflow here with the ID of the Release', + `run that published ${released}.`, + ].join('\n'), + }; + } + + if (compareVersions(released, documented) < 0) { + return { + behind: true, + report: [ + `The SDK has released ${released}, and the newest API reference this`, + `portal documents is ${documented}.`, + '', + "A release's reference is handed over separately from publishing it,", + 'and nothing fails when that step is missed. See "Publishing the API', + 'reference" in the SDK\'s `RELEASING.md`, or run the **SDK API sync**', + 'workflow here with the ID of the Release run that published', + `${released}.`, + ].join('\n'), + }; + } + + return { + behind: false, + report: + `The SDK has released ${released}, and this portal documents ` + + `${documented}.`, + }; +} diff --git a/build/shared/sdk-release.test.mts b/build/shared/sdk-release.test.mts new file mode 100644 index 000000000..f08a25027 --- /dev/null +++ b/build/shared/sdk-release.test.mts @@ -0,0 +1,93 @@ +/** + * @file Regression tests for reading SDK releases and reporting drift. + * @author The OpenINF Authors & Friends + * @license MIT OR Apache-2.0 OR BlueOak-1.0.0 + * @module {type ES6Module} build/shared/sdk-release.test + */ + +import { match, strictEqual } from 'node:assert/strict'; +import { test } from 'node:test'; +import { + driftReport, + newestRelease, + releaseOf, +} from '@openinf/portal/build/sdk-release'; + +test('reads the release out of a tag Changesets wrote', () => { + strictEqual(releaseOf('@openinf/util-types@2.1.3'), '2.1.3'); + strictEqual(releaseOf('@openinf/util@10.0.0'), '10.0.0'); + strictEqual(releaseOf('@openinf/util@3.0.0-rc.1'), '3.0.0-rc.1'); +}); + +test('reads a release tagged without a package name', () => { + strictEqual(releaseOf('v3.0.0'), '3.0.0'); + strictEqual(releaseOf('3.0.0'), '3.0.0'); +}); + +test('ignores a tag that does not name a release', () => { + // The scope's `@` is not the one the version follows, so a bare package + // name must not read as the release `openinf/util-types`. + strictEqual(releaseOf('@openinf/util-types'), undefined); + strictEqual(releaseOf('before-the-rewrite'), undefined); + strictEqual(releaseOf('@openinf/util@latest'), undefined); + strictEqual(releaseOf('@openinf/util@3.0'), undefined); + strictEqual(releaseOf(''), undefined); +}); + +test('chooses the newest release by precedence, not by name', () => { + strictEqual(newestRelease(['3.0.0', '10.0.0', '4.2.0']), '10.0.0'); + strictEqual(newestRelease(['3.0.0', '3.0.0-rc.1']), '3.0.0'); + strictEqual(newestRelease(['3.0.1']), '3.0.1'); + strictEqual(newestRelease([]), undefined); +}); + +test('reports nothing missing before the SDK has tagged a release', () => { + // The registry serves versions of every SDK package, published for years + // from repositories of their own. None of them had an artifact, so a + // portal documenting nothing is not behind them. + const { behind, report } = driftReport(undefined, undefined); + + strictEqual(behind, false); + match(report, /tagged no release/); +}); + +test('reports a portal that documents nothing once a release exists', () => { + const { behind, report } = driftReport('3.0.0', undefined); + + strictEqual(behind, true); + match(report, /released 3\.0\.0/); + match(report, /documents no\nrelease at all/); + match(report, /SDK API sync/); +}); + +test('reports a portal left behind by a newer release', () => { + const { behind, report } = driftReport('3.1.0', '3.0.0'); + + strictEqual(behind, true); + match(report, /released 3\.1\.0/); + match(report, /documents is 3\.0\.0/); + match(report, /RELEASING\.md/); +}); + +test('reports nothing when the portal documents the newest release', () => { + const { behind, report } = driftReport('3.0.0', '3.0.0'); + + strictEqual(behind, false); + match(report, /released 3\.0\.0, and this portal documents 3\.0\.0/); +}); + +test('reports nothing when the portal is ahead of the newest tag', () => { + // A release whose tag has not appeared yet, or a reference imported before + // the tag was pushed. Either way there is nothing missing here. + strictEqual(driftReport('3.0.0', '3.1.0').behind, false); +}); + +test('compares releases by precedence rather than by name', () => { + strictEqual(driftReport('10.0.0', '4.2.0').behind, true); + strictEqual(driftReport('4.2.0', '10.0.0').behind, false); +}); + +test('treats a release as newer than its own prereleases', () => { + strictEqual(driftReport('3.0.0', '3.0.0-rc.1').behind, true); + strictEqual(driftReport('3.0.0-rc.1', '3.0.0').behind, false); +}); diff --git a/build/shared/sdk-unpack.test.mts b/build/shared/sdk-unpack.test.mts new file mode 100644 index 000000000..2418bba1e --- /dev/null +++ b/build/shared/sdk-unpack.test.mts @@ -0,0 +1,292 @@ +/** + * @file Regression tests for placing an SDK release under `vendor/sdk-api/`. + * @author The OpenINF Authors & Friends + * @license MIT OR Apache-2.0 OR BlueOak-1.0.0 + * @module {type ES6Module} build/shared/sdk-unpack.test + * + * The task under test is the step between a downloaded artifact and the + * import the portal build runs, so what is checked here is what cannot be + * checked afterwards: that an existing release is never overwritten, that a + * directory name is a release version, and that nothing arrives by link. + */ + +import { + doesNotReject, + match, + ok, + rejects, + strictEqual, +} from 'node:assert/strict'; +import { execFile } from 'node:child_process'; +import { + mkdir, + mkdtemp, + readFile, + rm, + symlink, + writeFile, +} from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { test } from 'node:test'; +import { fileURLToPath } from 'node:url'; +import { promisify } from 'node:util'; + +const execute = promisify(execFile); +const task = fileURLToPath( + new URL('../tasks/unpack-sdk-api-artifact.mts', import.meta.url) +); + +/** The manifest a release directory carries, as the SDK's packager writes it. */ +const manifestFor = (version: string) => ({ + schemaVersion: 1, + version, + commit: '0123456789abcdef', + generatedAt: '2026-09-07T00:00:00.000Z', + docsPath: 'docs', + navigationPath: 'docs/navigation.json', +}); + +/** + * Creates a disposable workspace holding a downloaded artifact, and runs the + * real command in it. + * @param {string} version The release the download claims to be. + * @returns Fixture paths, runner, and cleanup for one isolated test. + */ +async function fixture(version = '3.0.0') { + const cwd = await mkdtemp(join(tmpdir(), 'sdk-unpack-test-')); + const download = join(cwd, 'download'); + const release = join(download, version); + + await mkdir(join(release, 'docs'), { recursive: true }); + await mkdir(join(cwd, 'vendor/sdk-api'), { recursive: true }); + await writeFile( + join(release, 'manifest.json'), + JSON.stringify(manifestFor(version)) + ); + await writeFile(join(release, 'docs/navigation.json'), '[]'); + await writeFile(join(release, 'docs/README.md'), '# API\n'); + + return { + cwd, + download, + release, + vendored: join(cwd, 'vendor/sdk-api', version), + run: (output?: string) => { + // Actions sets `GITHUB_OUTPUT` for every step it runs, so a test that + // left it alone would append to the real workflow's outputs when this + // suite runs in CI. Each test says what it should be, or that there + // should be none. + const env = { ...process.env }; + delete env.GITHUB_OUTPUT; + if (output !== undefined) env.GITHUB_OUTPUT = output; + + return execute(process.execPath, [task, '--from', download], { + cwd, + env, + }); + }, + cleanup: () => rm(cwd, { recursive: true, force: true }), + }; +} + +test('places a release and says what it added', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + const { stdout } = await f.run(); + + match(stdout, /Added release 3\.0\.0/); + strictEqual( + await readFile(join(f.vendored, 'docs/README.md'), 'utf8'), + '# API\n' + ); + strictEqual( + JSON.parse(await readFile(join(f.vendored, 'manifest.json'), 'utf8')) + .version, + '3.0.0' + ); +}); + +test('reports the release it added for the workflow to name a branch', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + const output = join(f.cwd, 'output.txt'); + await writeFile(output, ''); + await f.run(output); + + strictEqual(await readFile(output, 'utf8'), 'versions=3.0.0\n'); +}); + +test('says nothing to a caller that is not a workflow', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + // The variable is unset rather than empty outside a workflow, and an + // unconditional append would write `versions=` to a file named by nothing. + await doesNotReject(f.run('')); +}); + +test('refuses to replace a release it already documents', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + await f.run(); + await writeFile(join(f.vendored, 'docs/README.md'), '# Edited\n'); + + await rejects(f.run(), (error: { stderr: string }) => { + match(error.stderr, /already here/); + match(error.stderr, /adds a version rather than replacing one/); + return true; + }); + + // Refused before anything was written, rather than partway through it. + strictEqual( + await readFile(join(f.vendored, 'docs/README.md'), 'utf8'), + '# Edited\n' + ); +}); + +test('refuses a directory that is not named for a release', async (t) => { + const f = await fixture('latest'); + t.after(f.cleanup); + + await rejects(f.run(), (error: { stderr: string }) => { + match(error.stderr, /latest is not a release version/); + return true; + }); +}); + +test('refuses a release directory that would climb out of the vendor tree', async (t) => { + const cwd = await mkdtemp(join(tmpdir(), 'sdk-unpack-test-')); + t.after(() => rm(cwd, { recursive: true, force: true })); + + const download = join(cwd, 'download'); + await mkdir(join(download, '..3.0.0'), { recursive: true }); + await mkdir(join(cwd, 'vendor/sdk-api'), { recursive: true }); + + await rejects( + execute(process.execPath, [task, '--from', download], { cwd }), + (error: { stderr: string }) => { + match(error.stderr, /is not a release version/); + return true; + } + ); +}); + +test('refuses an artifact that arrives by symbolic link', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + const secret = join(f.cwd, 'secret.txt'); + await writeFile(secret, 'not the release\n'); + await symlink(secret, join(f.release, 'docs/leaked.md')); + + await rejects(f.run(), (error: { stderr: string }) => { + match(error.stderr, /is a symbolic link/); + return true; + }); + + await rejects(readFile(join(f.vendored, 'docs/README.md'), 'utf8')); +}); + +test('refuses a manifest that describes a different release', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + await writeFile( + join(f.release, 'manifest.json'), + JSON.stringify(manifestFor('9.9.9')) + ); + + await rejects(f.run(), (error: { stderr: string }) => { + match(error.stderr, /describes "9\.9\.9"/); + return true; + }); +}); + +test('refuses a manifest written for a schema it cannot read', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + await writeFile( + join(f.release, 'manifest.json'), + JSON.stringify({ ...manifestFor('3.0.0'), schemaVersion: 2 }) + ); + + await rejects(f.run(), (error: { stderr: string }) => { + match(error.stderr, /schema version 2/); + return true; + }); +}); + +test('refuses a release with no manifest at all', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + await rm(join(f.release, 'manifest.json')); + + await rejects(f.run(), (error: { stderr: string }) => { + match(error.stderr, /manifest\.json could not be read/); + return true; + }); +}); + +test('refuses a download that holds no release', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + await rm(f.release, { recursive: true }); + + await rejects(f.run(), (error: { stderr: string }) => { + match(error.stderr, /holds no release directory/); + return true; + }); +}); + +test('refuses a download directory that is not there', async (t) => { + const cwd = await mkdtemp(join(tmpdir(), 'sdk-unpack-test-')); + t.after(() => rm(cwd, { recursive: true, force: true })); + + await rejects( + execute(process.execPath, [task, '--from', join(cwd, 'absent')], { cwd }), + (error: { stderr: string }) => { + match(error.stderr, /could not be read/); + return true; + } + ); +}); + +test('asks for a download to unpack when given none', async (t) => { + const cwd = await mkdtemp(join(tmpdir(), 'sdk-unpack-test-')); + t.after(() => rm(cwd, { recursive: true, force: true })); + + await rejects( + execute(process.execPath, [task], { cwd }), + (error: { stderr: string }) => { + match(error.stderr, /Pass --from/); + return true; + } + ); +}); + +test('places every release a download holds', async (t) => { + const f = await fixture(); + t.after(f.cleanup); + + const second = join(f.download, '3.1.0'); + await mkdir(join(second, 'docs'), { recursive: true }); + await writeFile( + join(second, 'manifest.json'), + JSON.stringify(manifestFor('3.1.0')) + ); + await writeFile(join(second, 'docs/README.md'), '# API\n'); + + const { stdout } = await f.run(); + + match(stdout, /Added releases 3\.0\.0, 3\.1\.0/); + ok( + await readFile(join(f.cwd, 'vendor/sdk-api/3.1.0/docs/README.md'), 'utf8') + ); +}); diff --git a/build/tasks/check-sdk-api-drift.mts b/build/tasks/check-sdk-api-drift.mts new file mode 100644 index 000000000..e7ea8fcb0 --- /dev/null +++ b/build/tasks/check-sdk-api-drift.mts @@ -0,0 +1,166 @@ +/** + * @file Compare the API reference published here against what the SDK tagged. + * @author The OpenINF Authors & Friends + * @license MIT OR Apache-2.0 OR BlueOak-1.0.0 + * @module {type ES6Module} build/tasks/check-sdk-api-drift + * + * Outside `verify/` for the reason `check-vendored.mts` is: it reaches the + * network, and every task in that directory runs on every pull request. A + * host that is slow or unreachable would fail changes that have nothing to do + * with it. + * + * Handing a release's documentation over is a step of its own, and nothing + * fails when it is skipped -- the packages are on the registry either way and + * this portal simply never hears about them. That silence is what this reads. + * + * It asks the SDK repository for its tags rather than asking the registry for + * versions, because the question is narrower than "what is published". Every + * package in the SDK was published for years from a repository of its own, + * and those versions are on the registry still; none was built with an + * artifact for this portal to import, so the registry would report a + * reference missing for releases that never had one. `changeset publish` + * tags what it publishes, in the SDK's own repository, so a tag there is + * exactly a release that shipped from the workspace the Release workflow + * builds the artifact from. + * + * What to make of the answer is in `build/shared/sdk-release.mts`, where it + * can be read without a network. This is the part that cannot: the tags, and + * the directories in this checkout. + * + * The exit code carries both answers at once, the way `check-vendored.mts` + * does: bit 1 is set when the portal is behind, bit 2 when the tags could not + * be read. Being behind is a thing to act on and an unreachable host is not, + * so neither hides the other. + */ + +import { readdir } from 'node:fs/promises'; +import { + driftReport, + newestRelease, + releaseOf, +} from '@openinf/portal/build/sdk-release'; + +/** Where releases are kept, relative to the repository root. */ +const ARTIFACTS = 'vendor/sdk-api'; + +/** The repository whose releases this portal documents. */ +const SDK = 'OpenINF/sdk'; + +/** How long to wait on the API before giving up, in milliseconds. */ +const TIMEOUT = 30_000; + +/** + * Tags to ask for at once, and how many pages to walk. Changesets tags every + * package it publishes, so one release of the SDK leaves as many tags as it + * has packages, and this list grows a good deal faster than the releases in + * it. The ceiling is high enough to be years away and low enough that a + * misbehaving API cannot hold the job open indefinitely. + */ +const PER_PAGE = 100; +const PAGES = 20; + +/** Bits of the exit code. Both can be set; neither masks the other. */ +const MATCHED = 0; +const BEHIND = 1; +const UNCHECKED = 2; + +/** + * Says what went wrong in a sentence rather than a stack trace. + * @param {unknown} error Whatever was thrown. + * @returns {string} Its message. + */ +const reasonOf = (error: unknown) => + error instanceof Error ? error.message : String(error); + +/** + * The releases this portal documents, read from the directories themselves + * rather than from the generated collection, which is not in the checkout. + * @returns {Promise} Every release version found. + */ +async function imported() { + try { + const entries = await readdir(ARTIFACTS, { withFileTypes: true }); + return entries.filter((entry) => entry.isDirectory()).map((e) => e.name); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return []; + throw error; + } +} + +/** + * Every release the SDK repository has tagged. + * + * The API does not promise an order for tags, so they are all read rather + * than the first page being trusted to hold the newest. A page short of + * `PER_PAGE` is the last one; running out of pages first is reported rather + * than answered, since the newest release could be on a page never asked for. + * @returns The releases found, or why they could not be read. + */ +async function tagged(): Promise<{ releases?: string[]; problem?: string }> { + const token = process.env.GITHUB_TOKEN; + const releases: string[] = []; + + for (let page = 1; page <= PAGES; page += 1) { + const url = + `https://api.github.com/repos/${SDK}/tags` + + `?per_page=${PER_PAGE}&page=${page}`; + let tags: { name?: unknown }[]; + + try { + const response = await fetch(url, { + headers: { + accept: 'application/vnd.github+json', + // The API refuses a request that does not introduce itself. + 'user-agent': `${SDK} API reference drift check`, + 'x-github-api-version': '2022-11-28', + ...(token ? { authorization: `Bearer ${token}` } : {}), + }, + signal: AbortSignal.timeout(TIMEOUT), + }); + + if (!response.ok) throw new Error(`HTTP ${response.status}`); + + tags = (await response.json()) as { name?: unknown }[]; + } catch (error) { + return { problem: reasonOf(error) }; + } + + if (!Array.isArray(tags)) { + return { problem: 'the API served something other than a list of tags' }; + } + + for (const tag of tags) { + if (typeof tag.name !== 'string') continue; + const release = releaseOf(tag.name); + if (release !== undefined) releases.push(release); + } + + if (tags.length < PER_PAGE) return { releases }; + } + + return { + problem: + `more than ${PER_PAGE * PAGES} tags, which is past what this reads. ` + + 'The newest release may be on a page it never asked for.', + }; +} + +const answer = await tagged(); +const { behind, report } = driftReport( + answer.releases === undefined ? undefined : newestRelease(answer.releases), + newestRelease(await imported()) +); + +// Everything goes to stdout, including what went wrong: whatever runs this +// keeps only that, and a reason written anywhere else is a reason lost. +if (answer.problem !== undefined) { + console.log(`The SDK's tags could not be read: ${answer.problem}`); + console.log(''); +} else { + console.log(report); +} + +process.exitCode = + MATCHED | + (behind ? BEHIND : MATCHED) | + (answer.problem === undefined ? MATCHED : UNCHECKED); diff --git a/build/tasks/unpack-sdk-api-artifact.mts b/build/tasks/unpack-sdk-api-artifact.mts new file mode 100644 index 000000000..50ae82377 --- /dev/null +++ b/build/tasks/unpack-sdk-api-artifact.mts @@ -0,0 +1,207 @@ +/** + * @file Place a downloaded SDK API artifact under `vendor/sdk-api/`. + * @author The OpenINF Authors & Friends + * @license MIT OR Apache-2.0 OR BlueOak-1.0.0 + * @module {type ES6Module} build/tasks/unpack-sdk-api-artifact + * + * The SDK's Release workflow attaches one `sdk-api-docs` artifact to the run + * that published, holding a single directory named for the release. This puts + * that directory where `compile.importSdkApiDocs` will find it. + * + * Outside `verify/` because it writes: every task in that directory runs on + * every pull request, and none of them changes the checkout. + * + * What it enforces is placement, not content. The import task validates the + * manifest, confines every path the manifest names, and refuses a corpus it + * cannot map onto portal URLs; running it is the next step of the workflow + * that runs this one, and the pull request this produces is checked by it + * again. Three rules cannot wait for that: + * + * - A release adds a version rather than replacing one, so that a link into + * an older reference keeps describing the release it was published with. + * By the time the import task reads a directory, an overwrite has already + * happened. + * - A directory name becomes a path here and a public URL later, so it has + * to be a release version and nothing else. + * - A symbolic link is resolved by whoever reads it. The import task refuses + * one pointing outside the artifact, but a link that stays inside is still + * a file in `vendor/` whose bytes are somewhere else, which is not what + * "vendored" means here. + * + * Usage: node build/tasks/unpack-sdk-api-artifact.mts --from + */ + +import type { Dirent } from 'node:fs'; +import { appendFile, cp, lstat, readdir, readFile } from 'node:fs/promises'; +import { join, resolve } from 'node:path'; + +/** Where releases are kept, relative to the repository root. */ +const ARTIFACTS = 'vendor/sdk-api'; + +/** + * A release version, spelled as the portal's importer and the SDK's packager + * both spell it. Anything else is not a release, and -- since the name is + * joined onto a path and published as a URL -- is not something to find out + * about later. + */ +const VERSION = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/; + +/** Reads one `--flag value` pair out of the command line. */ +function flag(name: string) { + const at = process.argv.indexOf(`--${name}`); + return at === -1 ? undefined : process.argv[at + 1]; +} + +/** Stops with a sentence rather than a stack trace. */ +function fatal(message: string): never { + console.error(message); + process.exit(1); +} + +/** + * Walks a downloaded directory and refuses anything that is not a regular + * file or a directory. `readdir` reports what a link points at rather than + * the link, so each entry is checked with `lstat` instead. + * @param {string} directory The tree to walk. + * @param {string} label How to name it in a refusal. + */ +async function refuseLinks(directory: string, label: string) { + for (const entry of await readdir(directory, { withFileTypes: true })) { + const here = join(directory, entry.name); + const named = `${label}/${entry.name}`; + const stats = await lstat(here); + + if (stats.isSymbolicLink()) { + fatal( + `${named} is a symbolic link. An artifact holds the release's own ` + + 'files; a link would vendor whatever it happens to point at on ' + + 'the day something reads it.' + ); + } + + if (stats.isDirectory()) await refuseLinks(here, named); + else if (!stats.isFile()) fatal(`${named} is not a regular file.`); + } +} + +/** + * Reads the manifest a release directory must carry, and checks it describes + * the directory it was found in. + * @param {string} directory The release directory. + * @param {string} version The release it claims to be, from its name. + */ +async function manifestOf(directory: string, version: string) { + const path = join(directory, 'manifest.json'); + let manifest: { schemaVersion?: unknown; version?: unknown }; + + try { + manifest = JSON.parse(await readFile(path, 'utf8')); + } catch (error) { + return fatal( + `${version}/manifest.json could not be read as JSON: ` + + `${error instanceof Error ? error.message : String(error)}` + ); + } + + if (manifest.schemaVersion !== 1) { + fatal( + `${version}/manifest.json declares schema version ` + + `${JSON.stringify(manifest.schemaVersion)}, which this portal does ` + + 'not know how to read.' + ); + } + + if (manifest.version !== version) { + fatal( + `${version}/manifest.json describes ` + + `${JSON.stringify(manifest.version)} rather than the release its ` + + 'directory is named for.' + ); + } +} + +const from = flag('from'); + +if (from === undefined) { + fatal( + 'Nothing to unpack. Pass --from , the directory the ' + + '`sdk-api-docs` artifact was downloaded into.' + ); +} + +const source = resolve(from); +let entries: Dirent[]; + +try { + entries = await readdir(source, { withFileTypes: true }); +} catch (error) { + fatal( + `${from} could not be read: ` + + `${error instanceof Error ? error.message : String(error)}` + ); +} + +const releases = entries.filter((entry) => entry.isDirectory()); + +// An artifact with nothing in it is the shape a failed download leaves +// behind, and copying it would report success having published no reference. +if (releases.length === 0) { + fatal( + `${from} holds no release directory. An \`sdk-api-docs\` artifact ` + + 'contains one directory, named for the release it documents.' + ); +} + +const added: string[] = []; + +for (const release of releases) { + const version = release.name; + + if (!VERSION.test(version)) { + fatal( + `${version} is not a release version. A directory here is named for ` + + 'the release it documents, because that name is also the URL the ' + + 'reference is published at.' + ); + } + + const destination = join(ARTIFACTS, version); + + // Checked before anything is read from the artifact, and before anything is + // written: a release adds a version rather than superseding one, and the + // reference a release shipped with is not this repository's to rewrite. + try { + await lstat(destination); + fatal( + `${destination} is already here. A release adds a version rather than ` + + 'replacing one, so that a link into it keeps describing the release ' + + 'it was published with. Remove that directory deliberately if it ' + + 'really is to be rebuilt.' + ); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error; + } + + const origin = join(source, version); + + await refuseLinks(origin, version); + await manifestOf(origin, version); + await cp(origin, destination, { recursive: true, verbatimSymlinks: true }); + + added.push(version); +} + +const list = added.join(', '); + +console.log( + `Added ${added.length === 1 ? 'release' : 'releases'} ${list} under ` + + `${ARTIFACTS}/. Run \`nps compile.importSdkApiDocs\` to check the ` + + 'portal can render it before committing.' +); + +// How the workflow that runs this names the branch and the pull request. It +// is the only value here a later step needs, and deriving it again by listing +// the directory would read whatever else happened to be there. +if (process.env.GITHUB_OUTPUT) { + await appendFile(process.env.GITHUB_OUTPUT, `versions=${list}\n`); +} diff --git a/package-scripts.yml b/package-scripts.yml index f2407efa2..110e07781 100644 --- a/package-scripts.yml +++ b/package-scripts.yml @@ -25,6 +25,10 @@ scripts: # Outside it for the same reason: this one reaches the network, and an # upstream that is slow or moved would fail unrelated changes. vendored: node build/tasks/check-vendored.mts + # Outside it for that reason too. Asks the SDK repository what it has + # released, and says whether a release's API reference never arrived + # here. The SDK API drift workflow runs it weekly. + sdkApiDrift: node build/tasks/check-sdk-api-drift.mts runtimes: node build/tasks/verify/verify-runtimes.mts svg: node build/tasks/verify/verify-svg.mts toml: node build/tasks/verify/verify-toml.mts @@ -48,6 +52,11 @@ scripts: # Squash a pull request into one commit with a message built from the # commits it contains. Takes a number, and --dry-run to see it first. land: node build/tasks/land-pull-request.mts + # Place a downloaded `sdk-api-docs` artifact under vendor/sdk-api/, where + # compile.importSdkApiDocs reads it. Takes --from . The SDK API + # sync workflow runs this; by hand it is step 2 of "Publishing the API + # reference" in the SDK's RELEASING.md. + unpackSdkApiArtifact: node build/tasks/unpack-sdk-api-artifact.mts build: nps compile.buildPortal test: nps verify.all start: eleventy --serve diff --git a/package.json b/package.json index ab75ff38f..a8ea6b3b5 100644 --- a/package.json +++ b/package.json @@ -15,6 +15,7 @@ "./build/landing": "./build/shared/landing.mts", "./build/pull-request": "./build/shared/pull-request.mts", "./build/sdk-docs": "./build/shared/sdk-docs.mts", + "./build/sdk-release": "./build/shared/sdk-release.mts", "./build/utils": "./build/utils.mts" }, "homepage": "https://open.inf.is", diff --git a/vendor/sdk-api/README.md b/vendor/sdk-api/README.md index 421ec2581..7876de640 100644 --- a/vendor/sdk-api/README.md +++ b/vendor/sdk-api/README.md @@ -35,9 +35,26 @@ vendor/sdk-api/ The SDK's **Release** workflow builds one on the run that publishes, from the versions that shipped and the commit they shipped from, and attaches it to that -run as **sdk-api-docs**. Adding it here is a deliberate step, described under -"Publishing the API reference" in the SDK's `RELEASING.md`: download the -artifact, unzip it beside the releases already here, and open a pull request. +run as **sdk-api-docs**. + +Bringing it here is the **SDK API sync** workflow, started with the ID of that +run. It downloads the artifact, places it beside the releases already here, +imports it to check this portal can render it, and opens a pull request. The +check on that pull request is what confirms the reference will appear; a +refused artifact fails the workflow instead of becoming one. + +This portal fetches rather than the SDK pushing. The SDK's release job holds +the credential that publishes to the registry, and giving it a second one that +could write here would widen what a mistake in that job reaches. Everything +needed to refuse a bad artifact is already here, so the fetching is here too. + +Nothing makes that step happen. When it is missed the packages are published +all the same and this portal simply never hears about them, so the **SDK API +drift** workflow asks the SDK weekly what it has released and files an issue +when a release is missing its reference here. + +By hand, the same three steps are described under "Publishing the API +reference" in the SDK's `RELEASING.md`. A release adds a version rather than replacing one. Every release keeps the reference it was published with, so a link into an older version keeps