Skip to content

Use handle-based reads in FileHelper - #5351

Merged
mnriem merged 13 commits into
github:mainfrom
mnriem:fix/file-helper-handle-reads
Nov 10, 2026
Merged

mnriem merged 13 commits into
github:mainfrom
mnriem:fix/file-helper-handle-reads

Conversation

@mnriem

@mnriem mnriem commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Use handle-based reads in FileHelper so path traversal, file-type checks, and reading operate on stable filesystem objects. Preserve supported contained symlinks, link chains, and trusted root aliases.

Extract the existing Windows handle primitive for reuse by FileHelper and catalog-cache I/O, add deterministic coverage for path replacement and handle cleanup, and update the FileHelper design contract. Narrow cleanup-fallback test fixtures to disable directory removal capabilities without disabling project reads.

Testing

Set up this checkout's .venv and editable install using the local development guide. Run commands from the checkout root.

  • Tested locally with .venv/bin/specify --help (Windows PowerShell: .\.venv\Scripts\specify.exe --help)
  • Ran relevant existing tests with .venv/bin/python -m pytest tests -q (Windows PowerShell: .\.venv\Scripts\python.exe -m pytest tests -q); use affected test paths for focused checks and LC_ALL=en_US.UTF-8 on macOS
  • Tested with a sample project (if applicable)

Commands and results:

.venv/bin/specify --help
Passed.

LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest tests/test_file_helper.py tests/specify_cli/test_file_policy.py tests/specify_cli/integrations/test_file_changes.py tests/specify_cli/workflows/test_catalog_cache.py tests/specify_cli/presets/test_command_remove.py tests/specify_cli/integrations/test_command_switch.py -q
555 passed, 82 skipped.

LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest tests -q
14,841 passed, 136 skipped, 15 failed before narrowing the cleanup-fallback fixtures.
The subsequent focused run passes all 14 affected cleanup cases.
The remaining version-output formatting failure also reproduces with the original FileHelper implementation.

The remaining failure is tests/specify_cli/test_version_execution.py::TestWindowsUvToolEnvironment::test_non_windows_uv_tool_dry_run_still_previews_installer: terminal wrapping splits the long checkout interpreter path. No unrelated formatting change is included.

Native Windows execution was not available on the macOS host. Windows read-path wiring is covered with simulated handles; existing native Windows handle tests remain in the suite. A separate sample-project run is not applicable to this internal filesystem change.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Authored on behalf of @mnriem by GitHub Copilot using GPT-6.1 Sol (model ID: gpt-6.1-sol) in autonomous agent mode with runtime-default reasoning settings. AI assistance covered implementation, documentation, tests, automated checks, and this pull request description; no line-by-line human review is claimed.

Preserve supported contained links while opening reads through verified filesystem handles. Share Windows handle management, cover path replacement and descriptor cleanup, and document the read contract.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 9, 2026 22:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Windows branch reopens a pathname after validating its handle, allowing in-place reparse changes to redirect reads to an outside file.

4 open issues
What changed in this PR

This PR moves FileHelper reads toward handle-based filesystem access while preserving the existing contained-symlink policy and trusted root aliases.

Changes:

  • Add POSIX descriptor-relative traversal and Windows hierarchy pinning, with opened-file type checks.
  • Extract the Windows handle primitive for reuse by FileHelper and catalog-cache I/O.
  • Add replacement and cleanup coverage, narrow cleanup-fallback fixtures, and update the safety contract.

AI-generated review for the requester by GitHub Copilot (model/settings unavailable; autonomous analysis and suggested fixes).

File Description
tests/​test_file_helper.py Adds replacement, pinning, descriptor-cleanup, and unsupported-platform coverage.
tests/​specify_cli/​presets/​_skill_cleanup_helpers.py Disables descriptor-relative removal without disabling safe reads.
src/​specify_cli/​workflows/​_catalog_cache.py Reuses the extracted Windows handle helper.
src/​specify_cli/​file_helper.py Introduces handle-based reads and descriptor type checks.
src/​specify_cli/​_file_handles.py Provides the shared Windows handle primitive.
design/​file-helper.md Documents read guarantees separately from mutation limitations.

Review effort: Max (auto)

Comment thread src/specify_cli/file_helper.py Resolved
Comment thread src/specify_cli/file_helper.py Resolved
Comment thread src/specify_cli/file_helper.py Resolved
Comment thread tests/test_file_helper.py Resolved
@mnriem mnriem added the triage-must-have Verdict: high-value, important work for Spec Kit — do first label Nov 9, 2026
Read directly from duplicated Windows handles, retain search-only directory access, and close superseded POSIX parents. Extend coverage for handle ownership, permissions, descriptor limits, and native Windows behavior.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 9, 2026 23:13
mnriem and others added 2 commits November 9, 2026 17:22
Use integer null arguments required by _winapi.CreateFile on Python 3.11.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use Darwin's native search-only flag when a Python build omits its binding, with permission regressions covering the missing-constant case.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The native Windows tests fail before exercising the intended behavior because of invalid CreateFile arguments, with an additional failure caused by the nonempty-file symlink fixture.

3 open issues
4 resolved since last review

Review effort: Max (auto)

Comment thread tests/test_file_handles.py Resolved
Comment thread tests/test_file_handles.py Resolved
Comment thread tests/test_file_helper.py Resolved
Copilot AI auto review requested due to automatic review settings November 9, 2026 23:37
Copy an OS-generated reparse buffer onto an empty source file and cover the fixture's native calls and cleanup paths.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@mnriem
mnriem requested an auto review from Copilot and removed request for Copilot November 9, 2026 23:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Attribute-only Windows opens can convert pinned directories into junctions and redirect subsequent reads outside the root, a critical safety gap not covered by the native regression test.

1 open issue
2 previously missed issues

In code that hasn't changed since last review

High severity Prevent junction swaps via attribute-only directory opens

src/​specify_cli/​file_helper.py:176

An attribute-only FILE_WRITE_ATTRIBUTES open bypasses write-sharing checks, so converting an emptied, pinned directory into a junction can redirect the next full-path open outside the root. This conflicts with design/file-helper.md (“Safety limits and migration”); the native test only checks GENERIC_WRITE, not this attribute-only case. Open children relative to their pinned parent handles and add a native Windows regression for attribute-only junction conversion.

Medium severity Avoid requiring directory-list permission for pinned reads

src/​specify_cli/​file_helper.py:172

These directory pins introduce an extra Windows permission requirement: windows_handle requests FILE_LIST_DIRECTORY (0x1), so a user with traversal permission and read access to a known file, but no folder-listing permission, can no longer read it. The previous pathname read did not require enumeration, and this also affects bootstrap reads of .specify/init-options.json. Please use a non-enumerating directory pin that retains rename/reparse protection, with a native Windows ACL regression test.

AI-generated review for the requester by GitHub Copilot (model: unknown; autonomous mode; code analysis).

2 resolved since last review

Review effort: Max (auto)

mnriem and others added 3 commits November 10, 2026 09:41
Preserve the upstream exclusive-create implementation and tests while unifying the private Windows handle helper.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve traversal-only directory access and strengthen permission-revocation, attribute-only junction, and ACL coverage while retaining the merged exclusive-create behavior.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 10, 2026 15:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Windows exclusive-creation pathname-race warning is incomplete, and safe-read tests lack negative coverage for three required capabilities.

2 open issues
1 resolved since last review

Review effort: Max (auto)

Comment thread design/file-helper.md Resolved
Comment thread tests/test_file_helper.py Resolved
Check each required POSIX read capability independently and document the Windows exclusive-create pathname limitation.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 10, 2026 16:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

A critical risk of unbounded memory growth during repeated Windows reads requires human assessment before approval.

No open issues 🎉

2 resolved since last review

Review effort: Max (auto)

Keep structure and pointer declarations at module scope so repeated child opens reuse type identities. Cover successful and failed opens plus native repeated-read lifetime checks.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 10, 2026 18:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Eager Windows-only imports break CLI startup on POSIX Python builds without the optional _ctypes extension, even for specify --help.

1 open issue

Review effort: Max (auto)

Comment thread src/specify_cli/_file_handles.py Resolved
Keep POSIX startup and filesystem operations independent of optional _ctypes while retaining one set of Windows ABI type definitions through the import cache.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 10, 2026 18:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because the documentation states access requirements only for the root, while those rights are required for every resolved directory and can cause leaf reads to fail.

No open issues 🎉

1 resolved since last review

Review effort: Max (auto)

Retain lazy Windows ABI initialization and parent-relative reads alongside upstream text overwrite support. Preserve both test suites and their distinct traversal contracts.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 10, 2026 19:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active maintainability finding flags a third POSIX parent-descriptor walk with differing capability checks, requiring future traversal and cleanup fixes in three places.

1 open issue

Review effort: Max (auto)

Comment thread src/specify_cli/file_helper.py Resolved
Centralize directory capabilities, search flags, and descriptor ownership for reads, exclusive creation, and text overwrites. Preserve operation-specific leaf semantics and Windows paths.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 10, 2026 19:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive NtCreateFile/ctypes read-confinement path lacks native Windows validation of ABI, sharing, ACL, and reparse semantics that macOS simulations cannot establish.

No open issues 🎉

1 resolved since last review

Review effort: Max (auto)

@mnriem
mnriem merged commit 7166c82 into github:main Nov 10, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-must-have Verdict: high-value, important work for Spec Kit — do first

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants