Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,26 @@ npm run start -- --sbom-cache sboms --purl-file queries.txt

### SBOM Caching Workflow

GitHub.com (including Enterprise Managed Users) and `*.ghe.com` data-residency
hosts use the [asynchronous SBOM API](https://docs.github.com/en/enterprise-cloud@latest/rest/dependency-graph/sboms):
request generation, poll the returned report URL (`202` while processing), then
download the SPDX JSON from the `302` redirect. Generation, polling and download
share a two-minute deadline per repository. Polls wait at least one second and
honor `Retry-After`; downloads do not receive your GitHub token. Failures are
recorded per repository, as before, and collection continues.

The [synchronous cloud endpoint is removed November 13, 2026](https://github.blog/changelog/2026-05-12-synchronous-sbom-api-deprecated/).
There is no synchronous fallback for cloud errors. GHES retains the synchronous
endpoint: the official [GHES 3.19](https://docs.github.com/en/enterprise-server@3.19/rest/dependency-graph/sboms)
and [3.22](https://docs.github.com/en/enterprise-server@3.22/rest/dependency-graph/sboms)
references document only that flow. Selection uses the normalized API base URL
(`/api/v3` for GHES), so custom GHES URLs work even without `--ghes`.
Existing GHES custom-CA client settings are unchanged.

No cache migration is needed. The downloaded SPDX document and its packages are
stored in the same per-repository cache shape; offline searches, incremental
skips, and branch dependency-review behavior are unchanged.

1. First collection (populates cache progressively as it runs):

```bash
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"dev": "tsx src/cli.ts",
"lint": "eslint . --ext .ts --max-warnings=0",
"lint:fix": "eslint . --ext .ts --fix",
"test": "node --test dist/fixtures.test.js dist/cli.test.js dist/exclude-archived.test.js && node dist/test-fixture-match.js && node dist/test-branch-search.js"
"test": "node --test dist/fixtures.test.js dist/cli.test.js dist/sbomFetch.test.js dist/exclude-archived.test.js && node dist/test-fixture-match.js && node dist/test-branch-search.js"
},
"engines": {
"node": ">=18.0.0"
Expand Down
14 changes: 11 additions & 3 deletions src/octokit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { retry } from "@octokit/plugin-retry";
import chalk from "chalk";
import fs from "fs";
import https from "https";
import { setTimeout as sleep } from "node:timers/promises";

export interface OctokitFactoryOptions {
token: string;
Expand Down Expand Up @@ -88,13 +89,20 @@ export function createOctokit({ token, baseUrl, userAgentExtra, suppressSecondar
console.error(`Rate limit exceeded for ${opt.method} ${opt.url}. Not retrying.`);
return false;
},
onSecondaryRateLimit: async (retryAfter: number, _options: unknown) => {
onSecondaryRateLimit: async (retryAfter: number, options: { request?: { signal?: AbortSignal } }) => {
if (!suppressSecondaryRateLimitLogs) {
console.warn(chalk.grey(`Secondary rate limit hit. Pausing for ${retryAfter}s.`));
}
try { onSecondaryRateLimitHit?.(retryAfter); } catch { /* swallow callback errors */ }
await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
return true;
const signal = options.request?.signal;
try {
await sleep(retryAfter * 1000, undefined, { signal });
} catch (error) {
// Decline retries on cancellation without logging an expected abort as a hook failure.
if (signal?.aborted) return false;
throw error;
}
return !signal?.aborted;
}
}
});
Expand Down
12 changes: 6 additions & 6 deletions src/sbomCollector.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { createOctokit } from "./octokit.js";
import type { RepositorySbom, CollectionSummary, SbomPackage, Sbom, BranchDependencyDiff, DependencyReviewPackageChange } from "./types.js";
import { fetchRepositorySbom } from "./sbomFetch.js";
import type { RepositorySbom, CollectionSummary, SbomPackage, BranchDependencyDiff, DependencyReviewPackageChange } from "./types.js";
import * as semver from "semver";
import { readAll, writeOne } from "./serialization.js";
import { submitSnapshotIfPossible } from "./componentSubmission.js";
Expand Down Expand Up @@ -488,20 +489,19 @@ export class SbomCollector {

const fullName = `${org}/${repo}`;
try {
const resp = await this.octokit.request("GET /repos/{owner}/{repo}/dependency-graph/sbom", { owner: org, repo, headers: { Accept: "application/vnd.github+json" } });
const sbomWrapper = resp.data as { sbom?: Sbom };
const packages: SbomPackage[] = sbomWrapper?.sbom?.packages ?? [];
const { sbom, etag } = await fetchRepositorySbom(this.octokit, org, repo);
const packages: SbomPackage[] = sbom.packages ?? [];
return {
repo: fullName,
org,
enterprise: this.opts.enterprise,
retrievedAt: new Date().toISOString(),
sbom: sbomWrapper?.sbom,
sbom,
packages,
repoPushedAt: repoMeta?.pushed_at,
repoUpdatedAt: repoMeta?.updated_at,
defaultBranch: repoMeta?.default_branch,
etag: (resp.headers as Record<string, string | undefined>)["etag"]
etag
};
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
Expand Down
Loading
Loading