Skip to content

fix(analyzer): treat quoted $(printf ... | ...) as runtime data, not a parse limit - #1109

Open
Varshith-Kali wants to merge 4 commits into
NVIDIA:mainfrom
Varshith-Kali:fix-1107-printf-quoted-substitution
Open

Varshith-Kali wants to merge 4 commits into
NVIDIA:mainfrom
Varshith-Kali:fix-1107-printf-quoted-substitution

Conversation

@Varshith-Kali

Copy link
Copy Markdown
Contributor

Fixes #1107.

Problem

A POSIX shell printf whose argument is a double-quoted command substitution piping printf '%s' "$1" into another command made static_patterns_tool_misuse record static_parse_limit (printf_format / unsupported_syntax), so the file counted as partially inspected. This is the common quote-a-value-for-sh/cron idiom, for example cronq() { printf "'%s'" "$(printf '%s' "$1" | sed ...)"; }.

Root cause

The quoted-result branch of _static_printf_substitution failed whenever a quoted substitution started with printf and _has_literal_printf_arguments was false, even when the substitution was a pipeline whose output another command produces. _is_printf_substitution then treated that failure as a coverage gap, so the outer printf argument parse was marked limited.

Fix

  • Added _printf_invocation_has_following_operator, which walks the substitution with the existing shell word reader and reports whether a pipeline, command list, or redirection follows the printf words. A pipe quoted inside a printf argument ('a|b') does not count, and an unresolved boundary stays fail-closed.
  • The quoted-result branch of _static_printf_substitution now returns unresolved without a diagnostic in that case, and _is_printf_substitution no longer treats it as a bounded printf substitution, so the value is handled as runtime-dependent data, as for the echo and assignment variants. The same treatment applies to allowlisted wrappers (command/builtin/env).
  • A bare printf with a runtime argument stays fail-closed, and literal printf handling (static_parse_limit on valid Rust, Python and POSIX shell source leaves files partially inspected #694) is unchanged.

Tests

New tests/nodes/analyzers/test_static_parse_limit_issue_1107.py (11 tests): all 5 line variants from the issue (the two failing printf-inner cases plus the three passing controls), ledger-level COMPLETED assertions for the two failing cases and the verbatim cronq idiom, a #694 literal $(printf '\177ELF\001') test asserting the static value is still 0x7f ELF 0x01, and scope guards asserting a bare runtime-arg printf and a quoted pipe in an argument stay partial.

Verification

Varshith-Kali added a commit to Varshith-Kali/SkillSpector that referenced this pull request Nov 13, 2026
Backtick substitutions keep their runtime-command gap: a backtick
printf piped into another command is the runtime command itself, so
its failure to resolve statically is still a coverage gap. The
$(...) form is runtime-dependent data, as before. Fixes the two
test-unit failures on NVIDIA#1109 in
test_table_column_lookahead_preserves_runtime_command_gaps
(backtick printf 'x' | cat in a markdown table cell).

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Fixed the 2 test-unit failures on this head in fc47c50. Root cause: the pipeline treatment in the #1107 fix also applied to backtick substitutions, so a backtick printf 'x' | cat in a markdown table cell lost its runtime-command gap. The fix scopes that treatment to $(...) only: a backtick substitution keeps its gap, as the quoted-result branch already did. Verified locally: the 2 failing tests fail on 3482e9c and pass with the fix, the 11 tests in test_static_parse_limit_issue_1107.py still pass, the full markdown coverage file (154 tests) and the sibling static-parse-limit files (75 tests) are green, ruff check and format clean. CI is re-running on the new head.

@yashrajp22 yashrajp22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The quoted pipeline case is handled, but executable substitutions can now lose their coverage warning, and the new operator scan bypasses cancellation. Please keep the exemption scoped to proven data arguments and preserve the runtime check.

recognized, _ = _printf_invocation_arguments(
inner, check_runtime=runtime_check, diagnostics=invocation_diagnostics
)
if recognized and not backtick and _printf_invocation_has_following_operator(inner):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we limit this exemption to proven quoted data arguments? It also applies when the substitution is the command itself. For example, $(printf '%s%s' r 'm -rf /' | cat) supplies the command and all its arguments through word splitting. This early return drops the reconstruction diagnostic, while the outer-operand check has nothing left to inspect. Please retain the coverage warning for executable substitutions while allowing the quoted printf argument from the issue.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, fixed in bbf57b7. The exemption is now scoped to proven quoted data arguments: _is_printf_substitution and _static_printf_substitution only take it when the caller proves the substitution sits inside a double-quoted span, via the quoted_result flag and the quote-limits map for candidates the word reader starts on directly. An unquoted $(printf ... | cat) in command position keeps its reconstruction diagnostic, pinned by a new regression test using your rm -rf / example. The quoted case from #1107 still completes, and backtick substitutions keep their runtime-command gap.

"""
cursor = 0
while True:
word, cursor, limited = _next_shell_invocation_word(inner, cursor, lambda: None)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please pass the caller's runtime check through this scan. _is_printf_substitution can reach it with input larger than _PRINTF_STATIC_CHARS, and quoted words have no _SHELL_COMMAND_WORD_CHARS cap. Using lambda: None makes this added full pass ignore cancellation and the analyzer deadline while reading a long quoted argument.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bbf57b7. _printf_invocation_has_following_operator now takes the caller runtime check and passes it to the word reader instead of lambda: None, threaded through from every call site. Added a regression test with a 9000-character quoted argument that raises on cancellation.

Varshith-Kali added a commit to Varshith-Kali/SkillSpector that referenced this pull request Nov 16, 2026
The NVIDIA#1107 exemption treated any $(printf ... | ...) substitution as
runtime data, including one in command position: $(printf '%s%s' r
'm -rf /' | cat) supplies the command and its arguments through word
splitting, and the early return dropped the reconstruction diagnostic
while the outer-operand check had nothing left to inspect. Scope the
exemption to proven quoted data arguments: _is_printf_substitution and
_static_printf_substitution now take it only when the caller proves
the substitution sits inside a double-quoted span, via the
quoted_result flag and the quote-limits map for candidates the word
reader starts on directly. A bare substitution keeps its coverage
warning, and backtick substitutions keep their runtime-command gap.

Thread the caller's runtime check through the operator scan as well:
_printf_invocation_has_following_operator took lambda: None, so the
added full pass ignored cancellation and the analyzer deadline while
reading a long quoted argument, which has no _SHELL_COMMAND_WORD_CHARS
cap. It now receives the caller's check from every call site.

Addresses review feedback on NVIDIA#1109.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Decision — Request changes.

Value and readiness — The cron quoting idiom should complete static inspection when the runtime value is data. The current revision preserves the unquoted-command diagnostic and threads the runtime check through the added scan. It still treats quote provenance as proof of argument position.

Material findings

  • static_patterns_tool_misuse.py:2523–2541 allows the exemption when quoted_result is true. A shell can execute a double-quoted substitution as the command name. For example, "$(printf '%s' "$tool_name" | cat)" uses the resulting single word as the executable. The quoted-word reader and quote-limits fallback both take the exemption here, and there is no outer operand to trigger the fallback diagnostic. Require a proven non-command data position separately from quoting, and add the quoted-command counterpart to the existing unquoted-command regression. Keep the original outer-printf fixture completing. Exemption · Quoted word reader

PIC tradeoffs — A data-position requirement removes spurious coverage gaps for the original use case while keeping a coverage warning when the executable cannot be determined. The revised scan's cooperative cancellation preserves the existing deadline behavior.

Verification and gaps — Reviewed the complete diff, relevant parser paths, prior findings and replies, and the regression fixtures. All six recorded checks succeeded for this head. The PR remains conflicting, and the current executable-substitution fixture covers only the unquoted case. Static review only.

Reviewed head: bbf57b7212c161de715c529a6adb6d7b1f02f72f

…a parse limit

A POSIX shell printf whose argument is a double-quoted command
substitution piping printf '%s' "$1" into another command made
static_patterns_tool_misuse record static_parse_limit (printf_format /
unsupported_syntax), so the file counted as partially inspected. The
inner substitution's output comes from the downstream command, not
from printf, so it is runtime-dependent data in an argument position
of the outer printf, as for the echo and assignment variants.

Add _printf_invocation_has_following_operator, which walks the
substitution with the shell word reader and reports whether a
pipeline, command list, or redirection follows the printf words.
The quoted-result branch of _static_printf_substitution returns
unresolved without a diagnostic in that case, and
_is_printf_substitution no longer treats it as a bounded printf
substitution. A bare printf with a runtime argument stays
fail-closed, a pipe quoted inside a printf argument still does not
count as a pipeline, and literal printf handling (NVIDIA#694) is unchanged.

Fixes NVIDIA#1107

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Backtick substitutions keep their runtime-command gap: a backtick
printf piped into another command is the runtime command itself, so
its failure to resolve statically is still a coverage gap. The
$(...) form is runtime-dependent data, as before. Fixes the two
test-unit failures on NVIDIA#1109 in
test_table_column_lookahead_preserves_runtime_command_gaps
(backtick printf 'x' | cat in a markdown table cell).

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
The NVIDIA#1107 exemption treated any $(printf ... | ...) substitution as
runtime data, including one in command position: $(printf '%s%s' r
'm -rf /' | cat) supplies the command and its arguments through word
splitting, and the early return dropped the reconstruction diagnostic
while the outer-operand check had nothing left to inspect. Scope the
exemption to proven quoted data arguments: _is_printf_substitution and
_static_printf_substitution now take it only when the caller proves
the substitution sits inside a double-quoted span, via the
quoted_result flag and the quote-limits map for candidates the word
reader starts on directly. A bare substitution keeps its coverage
warning, and backtick substitutions keep their runtime-command gap.

Thread the caller's runtime check through the operator scan as well:
_printf_invocation_has_following_operator took lambda: None, so the
added full pass ignored cancellation and the analyzer deadline while
reading a long quoted argument, which has no _SHELL_COMMAND_WORD_CHARS
cap. It now receives the caller's check from every call site.

Addresses review feedback on NVIDIA#1109.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
…xemption

A shell executes a double-quoted substitution in command position as the command name, so quoting alone never proves a data argument: "$(printf '%s' "$tool_name" | cat)" on its own line lost its coverage warning, with no outer operand left to trigger the fallback diagnostic. The pipeline exemption now also needs a proven non-command data position, via the file's own _is_shell_command_position test: the word reader computes it from the enclosing word's start, the outer-operand fallback checks the candidate word, and the quote-limits fallback receives it from the full source since a fragment alone proves nothing. A native argument is proven data by its caller, and the argument tokenizer only scans argument words. The position travels as a separate quoted_data_position flag so a literal printf in command position still resolves to its value and keeps its destructive-head finding. The original outer-printf fixture still completes, and the quoted-command case is pinned by a new regression test alongside the unquoted one.

Addresses rng1995's review on NVIDIA#1109.

Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
@Varshith-Kali
Varshith-Kali force-pushed the fix-1107-printf-quoted-substitution branch from bbf57b7 to 5f60df5 Compare November 19, 2026 16:47
@Varshith-Kali

Copy link
Copy Markdown
Contributor Author

Fixed in 5f60df5, and rebased onto current main (48e0f94), so the conflict is gone.

The exemption now needs a proven non-command data position on top of quoting, using the file's own _is_shell_command_position test. The word reader computes it from the enclosing word's start, the outer-operand fallback checks the candidate word, and the quote-limits fallback receives it from the full source, since a fragment alone proves nothing. A native argument is proven data by its caller, and the argument tokenizer only scans argument words.

The position travels as a separate quoted_data_position flag, so a literal printf in command position still resolves to its value and keeps its destructive-head finding. That split mattered: gating the shared quoted_result flag broke the TM1 HIGH finding for "$(printf "%s\n" rm)" -rf /, now restored.

Added the quoted-command counterpart to the unquoted regression: your "$(printf '%s' "$tool_name" | cat)" example, the rm variant, the quoted command printf wrapper case, and the same word after ; and after | all keep the coverage warning. The original outer-printf fixture still completes. Test files green: 864 passed, 1 skipped across the 1107, 901, literals, and static-patterns files.

One known limitation: a word after an assignment prefix (FOO=bar "$(...)") still reads as data, since the position test only sees separators and line starts. Say the word and I will tighten that too, it needs a small assignment-prefix walk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

static_parse_limit (printf_format): printf with a quoted $(printf '%s' "$1" | sed …) argument marks shell scripts partially inspected

3 participants