Skip to content

fix(skills): fall back to CDN mirrors when the catalog API is rate limited (0.5.1) - #7

Closed
MarkShawn2020 wants to merge 1 commit into
mainfrom
fix/catalog-rate-limit-fallback
Closed

MarkShawn2020 wants to merge 1 commit into
mainfrom
fix/catalog-rate-limit-fallback

Conversation

@MarkShawn2020

Copy link
Copy Markdown
Contributor

Problem

npx lovstudio@0.5.0 skills add wxmp-cli failed with 读取 Lovstudio Skills 目录失败:catalog request failed: HTTP 403. The catalog comes from the unauthenticated GitHub contents API, which allows 60 requests per hour per IP. Users behind a shared NAT use that up quickly, and the CLI won't install anything without a catalog.

Change

  • loadCatalog tries the sources in order and uses the first one that returns a valid catalog:
    1. api.github.com contents API. This is unchanged as the first choice so pushes still show up right away. It sends GITHUB_TOKEN / GH_TOKEN when one is set, and only to api.github.com.
    2. raw.githubusercontent.com, which can be about 5 minutes behind.
    3. cdn.jsdelivr.net, which can be up to about 12 hours behind.
  • When a fallback source is used, the CLI prints one line to stderr naming the source and the earlier failures.
  • The API request now uses the application/vnd.github.raw+json media type. The base64 JSON form comes back empty once a file passes 1 MB, and skills.yaml is already at 499 KB.
  • A response without a skills list now counts as a failure instead of an empty catalog. This covers a mirror that returns an HTML block page with status 200.
  • LOVSTUDIO_SKILLS_CATALOG_URL still replaces the whole chain.
  • The paid gate is unchanged: if every source fails, loadCatalog throws and both callers exit before installing.

Verification

  • npm test: 43 pass, 1 skip (already skipped), 0 fail. Two new tests:
    • The API returns 403 and the raw copy is used. The token is sent to the API and not to the mirror.
    • Every source fails (429, an HTML page, a DNS error), and the CLI rejects with a message that lists each failure.
  • Run against the real endpoints while this machine's IP had 0/60 anonymous requests left:
    • Anonymous: the API returned 403, the raw copy was used, and 222 skills loaded in 2.3 s.
    • With GH_TOKEN: the API answered directly and no fallback line was printed.
    • With the API at 403 and the raw URL at 404: the jsDelivr copy was used.
    • node bin/lovstudio.mjs skills add definitely-not-a-skill-xyz: the fallback line was printed, then "目录中没有找到 Skill", exit 2.

🤖 Generated with Claude Code

…mited (0.5.1)

Anonymous GitHub API calls are capped at 60 per hour per IP, so users
behind a shared NAT saw `skills add` fail with "catalog request failed:
HTTP 403". The catalog now tries the GitHub API (with GITHUB_TOKEN /
GH_TOKEN when set, sent only to api.github.com), then raw.githubusercontent.com,
then jsDelivr. The API uses the raw media type, avoiding the empty
base64 payload GitHub returns for files over 1 MB, and a response
without a skills list is treated as a failure rather than an empty
catalog. Installs still stop when no source returns the catalog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MarkShawn2020 added a commit that referenced this pull request Oct 29, 2026
…d Skills (0.5.2) (#9)

On links slower than about 50 KB/s, common between mainland China and
GitHub, `skills add` stopped before the entitlement check with
"读取 Lovstudio Skills 目录失败:The operation was aborted due to
timeout": the only catalog source was the GitHub API copy of skills.yaml,
about 740 KB of base64 JSON, under a 15 s timeout. Reproduced with 0.5.1
through a 30 KB/s proxy.

The catalog is now read from lovstudio.ai's /api/skills/catalog first
(about 20 KB, only name, runtime_name, paid and depends_on). If the site
fails, the CLI tries the GitHub API with the raw media type (sending
GITHUB_TOKEN / GH_TOKEN only to api.github.com), then raw.githubusercontent.com,
then jsDelivr. These fallbacks are carried over from the unmerged #7. A
response without a skills list counts as a failure, and nothing installs
when every source fails.

Once past the catalog, free Skills (including a paid Skill's free
dependencies, such as branding-consistency for koubo-media-creator) were
installed by cloning the whole lovstudio/skills repository: 117.7 MB in
3,826 files at HEAD, which slow links cannot finish within the installer's
300 s clone limit. The CLI now makes a partial (--filter=blob:none) sparse
clone of only the selected skills/<name> directories and installs from
that local copy, as it already does for paid Skills. Blobs arrive over
separate connections at checkout, so each git step is retried up to three
times; a clone whose checkout failed is removed first. `npx skills update`
skips local installs, so these Skills are updated by running `skills add`
again. `skills add skills` (every free Skill) keeps the full clone.

Through a 30 KB/s proxy, `skills add writing-style` (two free
dependencies) now finishes in 19 s, moving about 24 KB from the site and
490 KB from GitHub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MarkShawn2020

Copy link
Copy Markdown
Contributor Author

Superseded by #9, released in 0.5.2. Its GitHub API → raw → jsDelivr fallbacks were carried over, with lovstudio.ai's /api/skills/catalog now read first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant