Native gitignore Support - Open for Discussion

[steadytao]

Requested previously in:

• [Feature Request]: add .rsyncingore file support. #321
• Use the .gitignore file as information to exclude files #721

--filter=':- .gitignore' does not work. This reads .gitignore files as rsync per-directory merge filters. It does not implement Git ignore semantics and diverges on negation, anchoring, directory-only patterns, escaping and nested precedence.

git ls-files can provide an external solution when Git is installed and the source is a Git worktree. It does not provide native support for remote sources or ordinary directory trees containing .gitignore files.

I am not opposed to adding explicit native support. Before implementation, we should agree on the behaviour.

A possible interface would be an opt-in option such as --gitignore. It would interpret gitignore files on the sending side without silently changing the meaning of existing transfers.

The design needs to define:

• Whether nested gitignore files are supported and how their precedence is applied
• What anchored patterns are relative to when multiple source arguments are provided
• Whether only gitignore files are read or whether .git/info/exclude and core.excludesFile are also in scope
• How Git exclusions interact with existing rsync filter rules
• Whether ignored files are protected from receiver-side deletion, consistent with ordinary rsync exclusions
• Behaviour for remote sources and directories that are not Git worktrees
• Whether Git-compatible parsing is implemented internally or delegated to Git

Thoughts? Open to discussion and implementations.

[steadytao]

@xiaguangbo @Norlandz @Vinfall Bump.

[Vinfall]

From your questions it looks like it would open a can of worms if done improperly. To mitigate this, I think .gitignore support should not look for global config or .gitignore in parent directories, something like root = true in .editorconfig, if you happen to know that. Currently there is no global rsync config to define exclude list, .gitignore should remain the same.

Whether nested gitignore files are supported and how their precedence is applied

Based on my thoughts above, nested .gitignore files in parent directories should be ignored, but those in sub-directories could be merged. I'm not sure about the implementation detail, but it's worth checking how .editorconfig config are merged.

Whether only gitignore files are read or whether .git/info/exclude and core.excludesFile are also in scope

Only explicit .gitignore files are read, like how --exclude-from=FILE works.

How Git exclusions interact with existing rsync filter rules

Not sure about this. Personally I think it's even fine to make --gitignore and --exclude/--exclude-from mutually-exclusive. Additional excluding patterns are redundant if you can exclude files in .gitignore, but it can still be useful sometimes though (e.g., to exclude certain config from being overridden)

# without .gitignore support
rsync -av --update --delete --exclude=".vercel" --exclude=".venv" -exclude="config.py" rat/ rpi:~/projects/rat/

# with .gitignore support
# config.py here is a local config, which should not be overridden, other files are excluded in .gitignore
rsync -av --update --delete --exclude="config.py" rat/ rpi:~/projects/rat/

Behaviour for remote sources and directories that are not Git worktrees
Whether Git-compatible parsing is implemented internally or delegated to Git

Support it like how it works for .cvsignore, which does not require cvs cli to function. Rsync is a file-copying tool after all, .gitignore is just a exclude file with a different syntax. It should be fine to ignore certain functions that are not needed for file transfer.

[steadytao]

I agree that the first version should be bounded to .gitignore files at or below each explicit transfer root. It should not inspect parent directories, .git/info/exclude or core.excludesFile.

I would not make it mutually exclusive with rsync filter rules. Transfer-specific filters remain useful and the ordering must determine both selection and receiver-side deletion protection.

The smallest coherent contract seems to be:

• parse .gitignore files internally on the sending side;
• support nested files with Git-compatible precedence and negation;
• treat each explicit source argument as its own root;
• compose the resulting exclusions at one documented point in the rsync filter chain;
• protect ignored receiver files from deletion unless --delete-excluded is used; and
• work for remote sources and ordinary directory trees without requiring Git.

The remaining decision is exactly where these generated rules sit relative to explicit include, exclude and merge rules.