Repository navigation
fix(pe3): stop flagging credential words in Python docstrings and comments - #1145
Varshith-Kali wants to merge 3 commits into
Conversation
…ments PE3 raised HIGH findings on credential words mentioned in Python docstrings, comments, and message strings that read nothing: the demo skill in NVIDIA#1144 reported six HIGH findings, all from a module docstring, two guard docstrings, a comment, and a message string. Mentioning a credential path is not credential access, and the two guards are the clearest cases: rejecting /etc/passwd and refusing a symlink to ~/.ssh/id_rsa is what PE3 wants skills to do. PE3 now skips matches inside Python docstrings, bare string statements, and comments, and skips prose phrases (access the credentials and kin) inside any other string literal. Code patterns still fire on the path itself, so a literal that reaches a file API stays visible, as does a path assigned to a variable. Unproven source (windowed fragments, malformed modules) keeps every finding. Fixes NVIDIA#1144. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
The NVIDIA#1144 fix deliberately silences PE3 for credential paths named only in Python comments, so test_pe3_surrounding_warning_cannot_hide_ sensitive_path now fails: its fixture is two comment lines and nothing reads the path. That expectation is outdated, not the fix: the same warning comment followed by code that opens the path still flags PE3, and the new test_pe3_python_docstring_comment suite already pins the comment-only case as zero findings. Split the test in two. The comment-only case now asserts no PE3 finding, and the original intent (a surrounding warning cannot hide a sensitive path) is kept under its original name with a code line that actually reads ~/.ssh/id_rsa after the warning comment. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision — Request changes.
Value and readiness — Suppressing documented path mentions in genuine Python comments and docstrings addresses the reported PE3 false positives. The AST suppression must share the same source-ownership requirement as comment suppression.
Material findings
static_patterns_privilege_escalation.py:1250–1263calls_python_inert_string_spanseven whenpython_literal_spansreturnsNone. The fallback callsast.parse, which treats the shebang as a comment. For a.pyartifact with a shell shebang and a shell/Python polyglot body, a shell operation can lie inside what Python sees as a triple-quoted string, so the AST branch suppresses the credential-path match. The runner can still passfile_type="python"from the extension. Gate every suppression path on successful ownership proof for the complete artifact, and add a runner-level shell-shebang polyglot regression alongside the genuine docstring control. Suppression · Interpreter check
PIC tradeoffs — Keeping PE3 findings on unproven code preserves coverage of real executable surfaces. The tokenizer proof still permits the precision improvement for genuine Python without requiring credential-flow analysis.
Verification and gaps — Reviewed the complete diff, the static runner's file-type selection, tokenizer/AST helpers, and the PE3 fixtures. All six recorded checks succeeded for this head. The added malformed-Python test exercises a parse failure, while the polyglot case requires syntactically valid Python that fails the interpreter-ownership check. Static review only.
Reviewed head: cae25c57a61f5b5f27673dfe7bdea64c2c9ae35a
Move the _python_inert_string_spans call inside the branch where python_literal_spans is not None, so a .py artifact with a shell shebang and shell/Python polyglot content no longer gets its shell lines suppressed as inert strings. The fallback ast.parse treats the shebang as a comment, so unproven source wrongly hid real credential-path findings. Unproven source now keeps every finding, matching the comment suppression path and the class docstring. Add regression tests: a shell-shebang polyglot keeps its PE3 finding, and the same body under a Python shebang stays suppressed. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
|
Thanks, good catch, this was real. Verified on the previous head: for a .py artifact with a shell shebang and shell/Python polyglot content, python_literal_spans returns None (ownership unproven) but _python_inert_string_spans still returned a span covering the shell line, because its fallback ast.parse treats the shebang as a comment. The PE3 credential-path finding was wrongly suppressed. Fixed in 05661c0: the _python_inert_string_spans call in _PythonPe3Prose.suppresses (static_patterns_privilege_escalation.py) now runs only inside the branch where python_literal_spans is not None, so inert-string suppression shares the same source-ownership requirement as comment suppression. Unproven source keeps every finding, matching the class docstring. Added two regression tests in tests/nodes/analyzers/test_pe3_python_docstring_comment.py: a shell-shebang polyglot keeps its PE3 finding (fails without the fix, verified), and the same body under a Python shebang stays suppressed. Test file: 21 passed. |
Fixes #1144.
Problem
PE3 raised HIGH findings on credential words mentioned in Python docstrings, comments, and message strings that read nothing. The demo skill from the issue reports six HIGH findings on main (a09091e), all from a module docstring, two guard docstrings, a comment, and a message string. Mentioning a credential path is not credential access, and the two guards are the clearest cases: rejecting
/etc/passwdand refusing a symlink to~/.ssh/id_rsais what PE3 wants skills to do.Fix
In the PE3 loop, Python matches are now checked against proven string and comment spans before they become findings:
python_literal_spanstokenizer proof.access the credentialsand kin) inside any other string literal is skipped. It is a message, not an instruction to read a credential file.open("~/.ssh/id_rsa"),Path("credentials.json"),config_dir() / "credentials.json", and a path assigned to a variable.Tests
tests/nodes/analyzers/test_pe3_python_docstring_comment.py: 19 tests, including the issue's demo skill verbatim (0 PE3 findings, was 6 HIGH) plus docstring, comment, message-string, f-string, non-ASCII, and CRLF cases, and stay-visible cases for real file access.tests/nodes/analyzerssuite: 16065 passed with only those 2 tests failing before the test updates; the affected PE3 and static-pattern suites re-ran green after (1616 passed, 0 failed).skillspector scanof the demo skill reports 0 issues (was 6 PE3 HIGH). A control skill withopen("~/.ssh/id_rsa")andconfig_dir() / "credentials.json"still reports both as PE3 HIGH.Notes
The issue also asks whether a skill can declare its own credential file (for example in
SKILL.mdpermissions) so PE3 can tell it apart from reading someone else's credentials. That is a separate design question and is left out of this fix.