fix(sieve): resolve case-exact file probes and deduplicate contents (fixes #645)

CHANGELOG.md

@@ -692,6 +692,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   outcome now also lists the files kept because they had uncommitted user
   changes (`user_changes_present`). An apply that wrote some files and kept
   others reported only the written ones (#673).
+- Case-exact file resolution and deduplication on case-insensitive filesystems
+  (macOS APFS, Windows NTFS): `file_exists_handler`, `regex_handler`,
+  `llm_eval_handler`, `llm_extract_handler`, and config file discovery verify
+  path components against case-exact directory entries, preventing spurious
+  matches on lowercase aliases, deduplicating file contents read into
+  consultation evidence, and citing exact repository paths (fixes #645).
 - Control `when` and step `when` now use one evaluator
   (framework-design 5.1.1). A list-valued control `when` such as
   `{ ci_provider = ["github", "gitlab"] }` made the control N/A even when

docs/architecture/framework-design.md

@@ -425,6 +425,8 @@ files = ["SECURITY.md", ".github/SECURITY.md"]
 1. If any file in `files` matches → PASS
 2. If no file matches → FAIL
 
+File paths in `files` are resolved case-exactly on every filesystem; candidate paths that do not match the on-disk directory entry case verbatim are not matched.
+
 **Ceiling**: `{fail}`; `{pass, fail}` with `existence = true` (section 3.0.1). A file being present says nothing about its content, so a presence step concludes PASS only for a control whose requirement is literally that the file exists.
 
 #### Scenario: File found
@@ -543,6 +545,8 @@ These are the step type's declared `settings` and common step fields (section 3.
 2. No match -> INCONCLUSIVE, unless `fail_on_miss = true` -> FAIL
 3. No files resolved -> INCONCLUSIVE
 
+File resolution in `files` and `exclude_files` is case-exact across all filesystems, and repeated paths resolving to the same underlying physical file are deduplicated.
+
 Exclude mode (`exclude_files`) is unchanged: no files found -> PASS, files found -> FAIL, usually refined by `expr`.
 
 #### Scenario: Pattern match found
@@ -589,7 +593,7 @@ confidence_threshold = 0.8
 | `analysis_hints` | `list[str]` | Hints to guide analysis |
 | `confidence_threshold` | `float` | Passed to the model with the consultation. Since feature 041 a judgment's confidence is recorded with it but is never a decision input: a judgment never concludes a control. |
 
-**`files_to_include` resolution**: The handler MUST resolve `$FOUND_FILE` entries by looking up `found_file` in `context.gathered_evidence`. Each resolved file path MUST be read (up to 10KB per file, max 5 files) and included as `file_contents` in the `consultation_request`. File paths that are not absolute MUST be resolved relative to `context.local_path`. Files that cannot be read MUST be silently skipped. A file that resolves outside the repository MUST NOT be read and is recorded in the result's `skipped_outside_repository` (section 3.0.4).
+**`files_to_include` resolution**: The handler MUST resolve `$FOUND_FILE` entries by looking up `found_file` in `context.gathered_evidence`. Each resolved file path MUST be read (up to 10KB per file, max 5 files) and included as `file_contents` in the `consultation_request`. File paths that are not absolute MUST be resolved relative to `context.local_path`. Files that cannot be read MUST be silently skipped. A file that resolves outside the repository MUST NOT be read and is recorded in the result's `skipped_outside_repository` (section 3.0.4). Files to include are resolved case-exactly on every filesystem, and duplicate paths resolving to the same physical file are deduplicated before reading. Similarly, `llm_extract` resolves candidate files case-exactly on every filesystem and deduplicates physical files before gathering content.
 
 **Ceiling**: `{}`. The handler returns INCONCLUSIVE with a `consultation_request`; with `stop_on_llm` the control becomes `PENDING` with `pending.kind = "llm_judgment"` (section 5.4). A model judgment never concludes PASS. No promotion to conclude PASS exists for a model judgment.
 
@@ -2017,7 +2021,7 @@ The darnit framework package SHALL NOT contain code, modules, or string literals
 
 ### 11.1 use_locator for File Existence Checks
 
-A deterministic handler invocation MAY use `use_locator = true` instead of specifying `files` directly. When `use_locator = true` is set, the framework SHALL use the control's `locator.discover` list as the handler's `files` parameter.
+A deterministic handler invocation MAY use `use_locator = true` instead of specifying `files` directly. When `use_locator = true` is set, the framework SHALL use the control's `locator.discover` list as the handler's `files` parameter. Locator file discovery (`discover_files`) resolves candidate files case-exactly on every filesystem.
 
 - If the control has no `locator` configuration, the framework SHALL log a warning and return INCONCLUSIVE
 - If both `use_locator = true` and explicit `files` are specified, `files` takes precedence

packages/darnit-baseline/src/darnit_baseline/openssf-baseline.toml

@@ -4208,7 +4208,7 @@ help_md = """Create architecture/design documentation.
 
 [controls."OSPS-SA-01.01".locator]
 project_path = "documentation.design"
-discover = ["ARCHITECTURE.md", "docs/architecture.md", "docs/design.md", "DESIGN.md"]
+discover = ["ARCHITECTURE.md", "docs/ARCHITECTURE.md", "docs/architecture.md", "docs/design.md", "DESIGN.md"]
 kind = "file"
 
 [controls."OSPS-SA-01.01".locator.llm_hints]
@@ -4220,7 +4220,7 @@ look_for_urls = true
 handler = "pattern"
 # rst variants added; ARCHITECTURE/DESIGN commonly live as .rst
 # in Sphinx-based Python projects.
-files = ["ARCHITECTURE.md", "ARCHITECTURE.rst", "docs/architecture.md", "docs/architecture.rst", "docs/design.md", "docs/design.rst", "DESIGN.md", "DESIGN.rst", "README.md", "README.rst"]
+files = ["ARCHITECTURE.md", "ARCHITECTURE.rst", "docs/ARCHITECTURE.md", "docs/ARCHITECTURE.rst", "docs/architecture.md", "docs/architecture.rst", "docs/design.md", "docs/design.rst", "DESIGN.md", "DESIGN.rst", "README.md", "README.rst"]
 pass_if_any = true
 
 [controls."OSPS-SA-01.01".passes.pattern.patterns]
@@ -4236,8 +4236,8 @@ prompt = """Evaluate whether this design/architecture document describes real sy
 2. Does it describe actors (users, systems) that interact with the software?
 3. Does it explain data flows or interactions between components?
 A document with only TBD/TODO placeholders or generic headings without content MUST fail."""
-# Issue #402: mirror the sibling pattern's file list.
-files_to_include = ["ARCHITECTURE.md", "docs/architecture.md", "docs/design.md", "DESIGN.md", "README.md"]
+# Issue #402: include README.md and resolved files from sibling pattern pass.
+files_to_include = ["README.md", "$RESOLVED_FILES"]
 confidence_threshold = 0.7
 analysis_hints = [
     "Look for named components with descriptions of what they do",
@@ -4283,7 +4283,7 @@ help_md = """Document external APIs and interfaces.
 
 [controls."OSPS-SA-02.01".locator]
 project_path = "documentation.api"
-discover = ["API.md", "docs/api.md", "docs/api/", "openapi.yaml", "openapi.json", "swagger.yaml", "swagger.json"]
+discover = ["API.md", "docs/API.md", "docs/api.md", "docs/api/", "openapi.yaml", "openapi.json", "swagger.yaml", "swagger.json"]
 kind = "file"
 
 [controls."OSPS-SA-02.01".locator.llm_hints]
@@ -4296,7 +4296,7 @@ look_for_urls = true
 handler = "pattern"
 # README.rst added for Sphinx-based projects; API.rst / USAGE.rst
 # variants common in the same ecosystem.
-files = ["API.md", "API.rst", "docs/api.md", "docs/api.rst", "README.md", "README.rst", "docs/README.md", "USAGE.md", "USAGE.rst", "docs/getting-started.md", "openapi.yaml", "openapi.json", "swagger.yaml", "swagger.json"]
+files = ["API.md", "API.rst", "docs/API.md", "docs/API.rst", "docs/api.md", "docs/api.rst", "README.md", "README.rst", "docs/README.md", "USAGE.md", "USAGE.rst", "docs/getting-started.md", "openapi.yaml", "openapi.json", "swagger.yaml", "swagger.json"]
 pass_if_any = true
 
 [controls."OSPS-SA-02.01".passes.pattern.patterns]
@@ -4312,8 +4312,8 @@ prompt = """Evaluate whether this API/interface documentation describes real API
 2. Does it include parameter descriptions, request/response formats, or usage examples?
 3. Is this substantive documentation, not just a heading with no content?
 A document with only headings or TODO placeholders MUST fail."""
-# Issue #402: mirror the sibling pattern's file list (pick top 5 of 10).
-files_to_include = ["API.md", "docs/api.md", "README.md", "openapi.yaml", "swagger.json"]
+# Issue #402: include README.md and resolved files from sibling pattern pass.
+files_to_include = ["README.md", "$RESOLVED_FILES"]
 confidence_threshold = 0.7
 analysis_hints = [
     "Look for documented endpoints, methods, or CLI commands",

packages/darnit-baseline/src/darnit_baseline/remediation/orchestrator.py

@@ -1509,8 +1509,11 @@ def _recheck(
     from darnit.config.control_loader import load_controls_from_effective
     from darnit.config.merger import merge_configs
     from darnit.config.operator.loader import resolve_operator_config
+    from darnit.core.utils import clear_dir_entries_cache
     from darnit.tools import audit as audit_tools
 
+    clear_dir_entries_cache()
+
     framework = _get_framework_config()
     if framework is None:
         raise RuntimeError("the framework configuration could not be loaded")

packages/darnit/src/darnit/config/discovery.py

@@ -40,17 +40,27 @@ def discover_files(
     if file_locations is None:
         return {}
 
+    from darnit.core.utils import resolve_case_exact_path
+
     discovered = {}
 
     for ref_path, patterns in file_locations.items():
         for pattern in patterns:
-            full_pattern = os.path.join(local_path, pattern)
-            matches = glob_module.glob(full_pattern)
-            if matches:
-                # Use first match, relative path
-                rel_path = os.path.relpath(matches[0], local_path)
-                discovered[ref_path] = rel_path
-                break
+            if "*" in pattern or "?" in pattern:
+                full_pattern = os.path.join(local_path, pattern)
+                matches = glob_module.glob(full_pattern)
+                exact_matches = [m for m in matches if resolve_case_exact_path(m, local_path)]
+                if exact_matches:
+                    # Use first match, relative path
+                    rel_path = os.path.relpath(exact_matches[0], local_path)
+                    discovered[ref_path] = rel_path
+                    break
+            else:
+                exact = resolve_case_exact_path(pattern, local_path)
+                if exact and os.path.exists(exact):
+                    rel_path = os.path.relpath(exact, local_path)
+                    discovered[ref_path] = rel_path
+                    break
 
     return discovered
 

packages/darnit/src/darnit/core/utils.py

@@ -1,5 +1,6 @@
 """Shared utility functions for the baseline MCP server."""
 
+import functools
 import glob as glob_module
 import json
 import os
@@ -683,3 +684,103 @@ def get_git_ref(local_path: str) -> str | None:
     except (FileNotFoundError, OSError, subprocess.SubprocessError) as e:
         logger.debug(f"git ref command failed: {type(e).__name__}")
     return None
+
+
+@functools.lru_cache(maxsize=2048)
+def _dir_entries_for_mtime(dir_path: str, mtime_ns: int) -> frozenset[str]:
+    try:
+        return frozenset(os.listdir(dir_path))
+    except OSError:
+        return frozenset()
+
+
+def dir_entries(dir_path: str) -> frozenset[str]:
+    """Return directory entries as a frozenset, or empty frozenset on OSError.
+
+    Keys by directory path and st_mtime_ns so modifications (e.g. remediation file creation)
+    automatically invalidate the cache without requiring manual clearing.
+    """
+    try:
+        mtime_ns = os.stat(dir_path).st_mtime_ns
+    except OSError:
+        return frozenset()
+    return _dir_entries_for_mtime(dir_path, mtime_ns)
+
+
+def clear_dir_entries_cache() -> None:
+    """Clear cached directory listings (primarily for test environments)."""
+    _dir_entries_for_mtime.cache_clear()
+
+
+def resolve_case_exact_path(path: str, root: str | None = None) -> str | None:
+    """Verify that a path exists with case-exact directory entries.
+
+    On case-insensitive filesystems (such as macOS APFS or Windows NTFS),
+    `os.path.exists` or `open()` can succeed when probing a lowercase
+    path (e.g. `readme.md`) even if the repository only contains `README.md`.
+    This function traverses each path component and verifies against the
+    actual directory entries (`os.listdir`), ensuring exact-case matching
+    identical to case-sensitive filesystems (Linux).
+
+    Args:
+        path: File path (relative or absolute).
+        root: Optional repository root to anchor relative paths or bounds.
+
+    Returns:
+        The normalized absolute path if all components exist with exact case,
+        or None if any component is missing or mismatched in case.
+    """
+    if not path:
+        return None
+
+    if root is not None and not os.path.isabs(path):
+        full_path = os.path.normpath(os.path.join(os.path.abspath(root), path))
+        base = os.path.abspath(root)
+    else:
+        full_path = os.path.normpath(os.path.abspath(path))
+        if root is not None:
+            base_cand = os.path.abspath(root)
+            try:
+                if os.path.commonpath([base_cand, full_path]) == base_cand:
+                    base = base_cand
+                else:
+                    base = os.path.abspath(os.sep)
+            except ValueError:
+                base = os.path.abspath(os.sep)
+        else:
+            base = os.path.abspath(os.sep)
+
+    try:
+        rel = os.path.relpath(full_path, base)
+    except ValueError:
+        return None
+
+    if rel == ".":
+        return base
+    if rel == ".." or rel.startswith(".." + os.sep):
+        return None
+
+    curr = base
+    for part in rel.split(os.sep):
+        if not part or part == ".":
+            continue
+        if part not in dir_entries(curr):
+            return None
+        curr = os.path.join(curr, part)
+
+    return curr
+
+
+def get_file_identity(full_path: str) -> tuple[int, int] | str:
+    """Return a unique file identity (st_dev, st_ino) or realpath fallback.
+
+    Used to deduplicate candidate files (e.g. across case-aliases, symlinks,
+    or multiple probes pointing to the same underlying file) before reading
+    or reporting evidence.
+    """
+    try:
+        st = os.stat(full_path)
+        return (st.st_dev, st.st_ino)
+    except OSError:
+        return os.path.realpath(full_path)
+

packages/darnit/src/darnit/remediation/executor.py

@@ -511,11 +511,18 @@ def execute(
                 confirmation_required=needed.key,
             )
         if not dry_run:
+            from darnit.core.utils import clear_dir_entries_cache
+
+            clear_dir_entries_cache()
             result.plan = planned.plan
             result.approvals = self._used_approvals(planned.plan)
 
         if result.success and config.project_update and config.project_update.set:
             self._project_update(control_id, config.project_update, result)
+            if not dry_run:
+                from darnit.core.utils import clear_dir_entries_cache
+
+                clear_dir_entries_cache()
 
         return result
 
@@ -732,6 +739,9 @@ def _write(self, change: FileChange) -> FileChange:
         os.makedirs(os.path.dirname(full_path), exist_ok=True)
         _write_bytes_atomic(full_path, change.content.encode("utf-8"))
         manifest.record_file(repository, run_id, change.path, change.after_digest or "", checkout=self.local_path)
+        from darnit.core.utils import clear_dir_entries_cache
+
+        clear_dir_entries_cache()
         return change.model_copy(update={"ignored": True}) if ignored else change
 
     def amend_created(self, created: FileChange, content: str) -> FileChange: