Skip to content

fix(releases): scan BOM-marked UTF-16 files for secrets - #160

Merged
mrgoonie merged 2 commits into
nextlevelbuilder:mainfrom
dannyward630:fix/issue-105-utf16-secret-scan
Oct 29, 2026
Merged

mrgoonie merged 2 commits into
nextlevelbuilder:mainfrom
dannyward630:fix/issue-105-utf16-secret-scan

Conversation

@dannyward630

Copy link
Copy Markdown
Contributor

Summary

  • Decode and secret-scan BOM-marked UTF-16 files regardless of filename extension.
  • Keep the existing size cap by quarantining oversized UTF-16 files.
  • Add UTF-16LE and UTF-16BE regression coverage for x.reg.

Fixes #105

Validation

  • Focused release scanner suite: 74 passed.
  • pnpm typecheck: passed.
  • CLI build and version/help smoke checks: passed.
  • pnpm --filter web build: passed.
  • Full suite: 2,683 passed, 3 skipped; one unrelated collection failure in apps/web/tests/personal-kits/kit-enablement-smoke.test.ts (SyntaxError: Invalid or unexpected token).

Assisted by Codex (GPT-6).

dannyward630 and others added 2 commits October 29, 2026 08:39
ASCII behind a UTF-16 BOM decodes to valid CJK text, so scanning only the
decoded text let a plain-byte secret through. Scan both readings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mrgoonie

Copy link
Copy Markdown
Contributor

Maintainer follow-up (09de062): ASCII behind a UTF-16 BOM decodes to valid CJK, so scanning only the decoded text let a plain-byte secret through (e.g. FF FE + AKIA... in x.reg). The scan now unions the decoded-text and raw-byte readings; regression test added (fails on cfb17cb, passes now).

Local verification on 09de062 (rebased on main 25dabca): pnpm typecheck passed; pnpm test 142 files, 2691 passed / 3 skipped (the earlier kit-enablement-smoke collection failure does not reproduce). CI PR checks green.

@mrgoonie
mrgoonie marked this pull request as ready for review October 29, 2026 16:36
@mrgoonie
mrgoonie merged commit 268221c into nextlevelbuilder:main Oct 29, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(releases): secret scan skips UTF-16 BOM files outside text extensions

2 participants