Skip to content

feat: handle large diffs with a size budget - #42

Open
stn1slv wants to merge 4 commits into
m7medVision:mainfrom
stn1slv:feat/large-diff-budget
Open

stn1slv wants to merge 4 commits into
m7medVision:mainfrom
stn1slv:feat/large-diff-budget

Conversation

@stn1slv

@stn1slv stn1slv commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Problem

lazycommit commit and lazycommit pr send the whole diff to the model with no size limit. With a large change set, OpenAI rejects the request (context length or tokens-per-minute limit). Retry and model fallback then resend the same oversized request, and the user just sees an error.

What this changes

  • Diff.Compact(budget) in internal/domain: a diff that fits the budget is sent unchanged. A larger diff becomes a list of changed files with +/- line counts, followed by every file diff that fits in full. Leftover space goes to the first file that did not fit, cut at a line boundary and marked [truncated].
  • Lock files (go.sum, package-lock.json, yarn.lock, ...), minified files, source maps, and binary files are listed without their diffs.
  • New max_diff_chars setting in prompts.yaml, default 40000 (about 10k tokens). It layers like num_suggestions, so a repo-local lazycommit.prompts.yaml can override it. Values below 1000 are raised to 1000 so the file list always fits.
  • The git adapter now passes --no-color --no-ext-diff --src-prefix=a/ --dst-prefix=b/, so settings such as diff.noprefix, diff.mnemonicPrefix, diff.external, or color.ui=always cannot change the format the parser reads.

Behavior change: diffs larger than 40000 bytes are now shortened by default. If you use a large-context model, raise max_diff_chars. There is no "off" value yet. I can add one (for example -1) if you want it.

Testing

  • Unit tests for compaction (budget limit, generated and binary files, quoted and renamed paths, long file lists, input without git headers), config layering, the pipeline, and the git format flags.
  • go test ./internal/... and golangci-lint run pass.
  • On a real 385 KB diff from this repo's history, the compacted prompt was under 40 KB, listed all 50 files, and kept 17 full file diffs plus one truncated file.

Oversized diffs made OpenAI-compatible backends reject the request.
Compact keeps a file summary and as many file diffs as fit, and drops
lock, minified, and binary file bodies.
Settings such as diff.noprefix, diff.external, or color.ui=always
changed the output that the diff parser relies on.
The suggestion pipeline compacts every diff to this budget (default
40000) before building the prompt, for both commit and PR flows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant