You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(compression): dedupe large blocks by full content - #621
Large-block deduplication in the observation compression layer now compares full message content, so two messages that merely share a leading 200 characters are no longer collapsed into a false "same content" pointer.
Root Cause & Problem
deduplicateLargeBlocks in src/compression/layers/observation.ts used msg.content.slice(0, 200) as the block identity key. Any two long messages with the same first 200 characters were treated as identical, and the later one was replaced with [See message #N - same content]. The text after the prefix was discarded, even though it can carry distinct and important information. The emitted pointer also asserted "same content" when the content was not actually the same.
Technical Solution
Use the complete msg.content string as the map key. This keeps the map-based exact-match behavior and removes the prefix-only false positives, with no new dependencies. Added src/compression/layers/observation.test.ts with two cases: one asserts that distinct messages sharing the first 200 characters both survive compression, and one asserts that genuinely identical blocks are still collapsed. The first case fails on the original code with expected '[See message #1 - same content]' to contain 'BETA_UNIQUE'.
Test Plan
Commands run on Windows with Node 24.22.1:
npx vitest run src/compression/layers/observation.test.ts on the unfixed code: FAILED as expected, 1 failed / 1 passed, with AssertionError: expected '[See message #1 - same content]' to contain 'BETA_UNIQUE'.
npx prettier --check src/compression/layers/observation.ts src/compression/layers/observation.test.ts: exit code 0, All matched files use Prettier code style! (after prettier --write on the same two files).
npx eslint src/compression/layers/observation.ts src/compression/layers/observation.test.ts: exit code 0, no diagnostics.
npm run typecheck (tsc --noEmit): exit code 0, no errors.
npx vitest run src/compression/layers/observation.test.ts after the fix: PASS, 2 passed / 2 passed.
Only the two changed files were touched. No new dependencies were added. The full repository test suite was not run locally.
Summary by CodeRabbit
Bug Fixes
Distinct long messages that share the same first 200 characters are no longer mistakenly treated as duplicates. Truly repeated messages continue to be marked as identical.
No actionable comments were generated in the recent review.
Walkthrough
Large observation messages are now deduplicated by full content rather than by the first 200 characters. Tests cover distinct messages with a shared prefix and identical messages.
Changes: Observation deduplication
Layer / File(s)
Summary
Full-content matching and tests src/compression/layers/observation.ts, src/compression/layers/observation.test.ts
deduplicateLargeBlocks uses the full message content as its matching key. Tests verify that messages with a shared 200-character prefix but different tails remain distinct, while identical messages are marked "same content".
Large observation messages that share an opening but differ afterwards are no longer replaced by a "same content" pointer, so their text is preserved. Exact duplicates are still collapsed. The change is small, tested, and easy to revert, so merge risk is minimal.
Pre-merge checks: 4
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title names the change: deduplicating large blocks by full content in the compression layer. This matches the summary, which changes deduplicateLargeBlocks from a 200-character prefix key to the f…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the change: deduplicating large blocks by full content in the compression layer. This matches the summary, which changes deduplicateLargeBlocks from a 200-character prefix key to the full message content, and the objectives, which describe the same fix. The title is specific and concise, follows the conventional commit style, and identifies the primary change.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Reviewing files that changed from the base of the PR and between aa91d65 and 9758487.
Files selected for processing: 2
src/compression/layers/observation.test.ts
src/compression/layers/observation.ts
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Large-block deduplication in the observation compression layer now compares full message content, so two messages that merely share a leading 200 characters are no longer collapsed into a false "same content" pointer.
Root Cause & Problem
deduplicateLargeBlocksinsrc/compression/layers/observation.tsusedmsg.content.slice(0, 200)as the block identity key. Any two long messages with the same first 200 characters were treated as identical, and the later one was replaced with[See message #N - same content]. The text after the prefix was discarded, even though it can carry distinct and important information. The emitted pointer also asserted "same content" when the content was not actually the same.Technical Solution
Use the complete
msg.contentstring as the map key. This keeps the map-based exact-match behavior and removes the prefix-only false positives, with no new dependencies. Addedsrc/compression/layers/observation.test.tswith two cases: one asserts that distinct messages sharing the first 200 characters both survive compression, and one asserts that genuinely identical blocks are still collapsed. The first case fails on the original code withexpected '[See message #1 - same content]' to contain 'BETA_UNIQUE'.Test Plan
Commands run on Windows with Node 24.22.1:
npx vitest run src/compression/layers/observation.test.tson the unfixed code: FAILED as expected, 1 failed / 1 passed, withAssertionError: expected '[See message #1 - same content]' to contain 'BETA_UNIQUE'.npx prettier --check src/compression/layers/observation.ts src/compression/layers/observation.test.ts: exit code 0,All matched files use Prettier code style!(afterprettier --writeon the same two files).npx eslint src/compression/layers/observation.ts src/compression/layers/observation.test.ts: exit code 0, no diagnostics.npm run typecheck(tsc --noEmit): exit code 0, no errors.npx vitest run src/compression/layers/observation.test.tsafter the fix: PASS, 2 passed / 2 passed.Only the two changed files were touched. No new dependencies were added. The full repository test suite was not run locally.
Summary by CodeRabbit