Skip to content

buffer: stop clamping indexOf byteOffset to int32 - #67009

Open
vedchaudhari wants to merge 1 commit into
nodejs:mainfrom
vedchaudhari:buffer-indexof-large-offset
Open

vedchaudhari wants to merge 1 commit into
nodejs:mainfrom
vedchaudhari:buffer-indexof-large-offset

Conversation

@vedchaudhari

Copy link
Copy Markdown
Contributor

On buffers bigger than 2 GiB, indexOf(), lastIndexOf() and includes() change any byteOffset above 2 ** 31 - 1 to 2 ** 31 - 1, so the search starts in the wrong place:

const G = 2 ** 30;
const buf = Buffer.alloc(3 * G);
buf[2.25 * G] = 0x0a;
buf[2.75 * G] = 0x0a;

buf.indexOf(0x0a, 2.5 * G);     // main: 2415919104 (before the offset), this PR: 2952790016
buf.includes(0x0a, 2.9 * G);    // main: true, this PR: false
buf.lastIndexOf(0x0a, 2.6 * G); // main: -1, this PR: 2415919104

The int32 limit was added in 2016 (6c1e5ad), when buffers couldn't be bigger than 2 GiB. MAX_LENGTH has grown a lot since v14, but the limit was never updated. The C++ side already uses 64-bit offsets, so I raised the limit to Number.MAX_SAFE_INTEGER, which still keeps values like Infinity out. The negative return values from the issue were already fixed by #66325.

The new cases in test/pummel/test-buffer-indexof-large.js fail on main and pass with this change. Results on normal-size buffers are unchanged.

Tested on macOS arm64.

Note: #66590 edits the same comment, so whichever lands second will need a small rebase.

Fixes: #67008

Assisted-by: Claude. I reproduced the bug, reviewed the change and ran the tests myself.

bidirectionalIndexOf() clamped byteOffset to the int32 range, so on
buffers larger than 2 GiB an offset above 2 ** 31 - 1 started the
search at 2 ** 31 - 1 instead. indexOf() and includes() could then
match before the requested offset, and lastIndexOf() missed matches.
Offsets below -(2 ** 31) were clamped the same way.

The binding already takes an int64 offset, so clamp to the safe
integer range instead. This still keeps Infinity and other values
that int64 can't represent from reaching the binding.

Fixes: nodejs#67008
Refs: nodejs#66325
Assisted-by: claude:opus-5.5
Signed-off-by: vedchaudhari <vedc2853@gmail.com>
@nodejs-github-bot nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. needs-ci PRs that need a full CI run. labels Oct 26, 2026
@codecov

codecov Bot commented Oct 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.66%. Comparing base (3983092) to head (4fb3d72).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #67009      +/-   ##
==========================================
- Coverage   92.96%   90.66%   -2.30%     
==========================================
  Files         428      797     +369     
  Lines      195232   278262   +83030     
  Branches    30184    53512   +23328     
==========================================
+ Hits       181492   252286   +70794     
- Misses      13413    16340    +2927     
- Partials      327     9636    +9309     
Files with missing lines Coverage Δ
lib/buffer.js 99.74% <100.00%> (+2.90%) ⬆️

... and 500 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

buffer Issues and PRs related to the buffer subsystem. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

buffer: indexOf/includes clamp byteOffset to 2^31-1 and return a negative index on buffers over 2 GiB

2 participants