Skip to content

Deduplicate deletes git branch refs and reflogs because .git folders under the root are scanned as ordinary files #192

Open

Description

@matt-edmondson

What's wrong

FileScanner.ScanForFiles (FileDeduplicator/FileScanner.cs:71-103) walks every file under the root using WalkOptions (:38-43). That walk skips only reparse points; nothing excludes version-control metadata.

Inside .git, many small files have identical content:

  • every loose branch ref that points at the same commit (41 bytes, <sha>\n)
  • logs/HEAD and logs/refs/heads/<current branch>
  • the reflogs of branches created from the same commit

DuplicateScan.Run (DuplicateScan.cs:49) groups these files as duplicates. Deduplicator.DeleteDuplicates (Deduplicator.cs:138-191) then keeps the file with the shortest name (OrderByPreference, :123-124) and deletes the rest. On a refs directory, that deletes branches.

Reproduction (main @ ec30966)

mkdir -p ws/proj && cd ws/proj && git init -q -b main
echo hi > a.txt && git add . && git commit -qm init
git branch feature && git branch release && cd ..
echo y | FileDeduplicator Deduplicate -p .

The tool groups the files like this:

  • KEEP .git/refs/heads/main; DELETE .git/refs/heads/feature and .git/refs/heads/release
  • KEEP .git/logs/HEAD; DELETE .git/logs/refs/heads/main
  • KEEP .git/logs/refs/heads/feature; DELETE .git/logs/refs/heads/release

It reports "Deleted 4 file(s). Reclaimed 345 B" and exits 0. Afterwards git branch lists only main. If those branches had diverged, their tips would now be reachable only through the reflogs, and the same run deleted those too.

The normal use is pointing the tool at a projects folder, a home directory or a backup drive. That hits every repository in the tree, silently deleting branches to reclaim a few hundred bytes per repository.

The same bug exists in ktools dedup (ktsu-dev/KtsuTools#222).

Suggested fix / acceptance criteria

  • Skip version-control metadata directories during the walk: at least .git, .svn, .hg and .bzr.
    • Use a directory filter on the enumeration, not a substring match on paths.
    • A .git file (a worktree or submodule pointer) needs no special handling.
  • Mention the exclusion in the README, next to the empty-file rule.
  • Add a test with a temp tree containing .git/refs/heads/{a,b} with identical content, plus a real duplicate pair outside .git. Scan, DryRun and Deduplicate should report and delete only the pair outside .git, and leave the .git files untouched.

Activity

  1. matt-edmondson commented on Oct 20, 2026

    @matt-edmondson
    ContributorAuthor

    Triage


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions