Repository navigation
fix(analyzer): stop TypeScript templates from hitting parse limits - #990
bniladridas wants to merge 6 commits into
Conversation
bniladridas
left a comment
There was a problem hiding this comment.
Change walkthrough for the TypeScript template ownership path. Each thread explains the hunk and what to check.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision: Changes Requested
Value and readiness
The complete-source TypeScript path addresses #884's ordinary template-literal completeness problem, and the TSX mapping, bounded walks and conservative controls are useful. The new lexical proof still grants ownership to a real shell-execution template, so it is not ready to approve.
Material findings
Required — preserve the execution-surface guard for string spellings, src/skillspector/typescript_tokens.py:132-138 and 342-346. For example, this valid TypeScript uses Node's child-process module:
import * as cp from "node:child_\x70rocess";
cp["exec"](`${format(n)}`);The module specifier's string value is node:child_process, and the computed member is the ordinary exec method. However, skip_string skips the escape and compares the raw source substring to the module allowlist, so this import is not rejected. The "exec" token is also handled as a string; only bare identifier tokens are compared with _SHELL_EXECUTION_CALLS. The remaining tokens and balanced interpolation pass the walk, so the function returns ownership for the execution template. _has_shell_command_word_exhaustion then skips its opening backtick and interpolation candidates through template_source.owns, dropping the conservative parse-limit evidence that this same complex template requires on an execution surface. The JavaScript path checks parsed module values and statically named computed members; this TypeScript path does neither. Decode supported string/identifier spellings or fail closed when they cannot be proven, and preserve execution-member checks for computed string properties. Add escaped-module and computed-execution-member regressions through both the helper and inspection ledger; the existing plain module/import and direct-call controls do not cover this case.
PIC tradeoffs
Reducing false partial results is valuable, but ownership must retain the existing conservative behavior when the template can supply a shell command. Recognizing syntax without accounting for its string value is insufficient for that proof.
Verification and gaps
Reviewed pinned head de9e5a1d933c452b1ceeb116084139afab54925f: complete six-file diff, new scanner and all new/modified tests, surrounding template ownership and shell completeness paths, JavaScript execution guard, file mappings, #884 and its corrections, all review/thread/discussion records. There are no prior own marked reviews. Live head metadata still matches the inventory. Hosted lint, extension, DCO and Docker checks pass; unit CI reports 17,830 passed and 15 failed. Compared every failing identity with the hosted baseline run at e14c450b260958ed3ebd83c94c09c4b093e3e656 (run 41513800480): all 15 match exactly and concern existing PowerShell/quoted-printf tests. The new TypeScript module has no reported test failure, but the execution spelling above has no regression. This is a static ownership/candidate-suppression proof, not a runtime reproduction or complete scanner-verdict claim. No contributor code, tests, imports, builds, setup, hooks, regexes, scanner invocations or snippets were executed.
de9e5a1 to
e8b516b
Compare
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
e8b516b to
5645cbb
Compare
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists. The shared call list also carries the JavaScript keyword "function", which is a declaration keyword in TypeScript and must not suppress ownership of a benign template.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
5645cbb to
5778ef7
Compare
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists. The shared call list also carries the JavaScript keyword "function", which is a declaration keyword in TypeScript and must not suppress ownership of a benign template.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists. The shared call list also carries the JavaScript keyword "function", which is a declaration keyword in TypeScript and must not suppress ownership of a benign template.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
5778ef7 to
b15673f
Compare
|
Fixed in b15673f, the second commit on the rebased branch (it was 5778ef7 before the rebase onto upstream main 62c2fe8). Review finding: the string path compared raw bytes, so The string path no longer compares raw bytes. Regressions cover escaped module names (hex, unicode, braced unicode, escaped The branch also includes the earlier fix for On the rebased branch |
|
Ready for re-review at b15673f. The branch is now rebased onto upstream The execution-surface finding from the review on On |
yashrajp22
left a comment
There was a problem hiding this comment.
Source review of b15673fd3fb8beab3b01b5e0b449a290b3cfeabb against 62c2fe8622f580ac292d4078c3b71f790194dbbd found four remaining template-ownership gaps, detailed inline.
I inspected the changed source, affected callers and tests. A completed Greptile review of the same comparison raised two overlapping findings, which I checked against the source. This is source-only evidence: I have not run tests or scanner reproductions, or verified fresh wheels and sample scans. Behavioral verification and the full review remain incomplete.
f4b6491 to
9db61d0
Compare
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists. The shared call list also carries the JavaScript keyword "function", which is a declaration keyword in TypeScript and must not suppress ownership of a benign template.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
|
Commit hashes in my earlier comments and replies predate the rebase onto |
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Request changes at 9db61d04fe7b847c464fdf7938678e25652cb030.
Value and readiness: Complete-source TypeScript template ownership addresses the corrected #884 reproducer and its ordinary formatting/path cases. The previous own marked review's escaped-module/computed-execution-member reproducer is addressed by decoded comparisons, with helper and ledger controls. The four later human findings are also addressed for their reported forms: exact lowercase function is distinguished from Function; loader callee context survives (; postfix ++/-- are consumed together; and shared delimiter tracking rejects the reported missing/crossed delimiters. Remaining execution-guard bypasses keep this head unready.
Material findings:
- [P1, required]
src/skillspector/typescript_tokens.py:571-594recognizes identifiers only with the ASCII_IDENTIFIERpattern and accepts an unrecognized backslash through the punctuation fallback. Valid TypeScript\u0065val(`${payload.code}`);names the ordinaryevalfunction, but the scanner reads the backslash as punctuation andu0065valas an unrelated identifier. No execution_name('eval') check occurs; the balanced dynamic template is returned as owned._TypeScriptTemplateOwnershipthen exposes these spans through owns(), and static_patterns_tool_misuse.py:3431-3432 skips the template's conservative shell-command candidates. Decode valid identifier Unicode escapes before the execution guard, or return None when identifier escapes cannot be proven. Add escaped-identifier and escaped-member controls through both the helper and ledger. This is distinct from decoding string literals in module/property positions. - [P1, required]
src/skillspector/typescript_tokens.py:221-222allows a regex after every closing brace, without proving whether it closed a statement block or an expression. In validconst n = {} / eval(payload) / 2; const label = `${item.name}`;, the first slash is division after an object expression; the current walk instead skips/ eval(payload) /as a regex, so the eval guard is never visited and the later dynamic template gains ownership. The added postfix-increment regression closes one form of this ambiguity but not closing object/function expressions. Track sufficient brace context, or fail closed on ambiguous slash-after-brace syntax, and add an object-expression division regression alongside the genuine regex-after-control/block positives.
PIC tradeoffs: Reducing false partial results must preserve conservative coverage when runtime code execution remains possible. These fixes should keep the demonstrated benign forms owned while withholding ownership from lexical forms the scanner cannot distinguish safely.
Verification and gaps: Reviewed the entire seven-file diff, complete new scanner/tests, surrounding exact-head template ownership/candidate-suppression callers and file mappings, #884 including its corrected trigger/impact, all normalized discussion/review/thread bodies, and the prior rng1995 marked review plus four later human followups. Hosted CI run 42392193808 has this exact head_sha and succeeds; the unit log includes the new TypeScript module and reports 21,038 passed, 24 skipped, 258 deselected and 4 xfailed. All six reported checks succeeded. Existing regressions do not cover either remaining form. Findings are static lexical/control-flow proofs, not locally reproduced scanner-verdict claims. No contributor code, tests, imports, builds, setup, hooks, regex probes or snippets were executed.
yashrajp22
left a comment
There was a problem hiding this comment.
Source re-review of 9db61d04fe7b847c464fdf7938678e25652cb030 against e48d771a26f7398b36256ceb8d8f94adce3d5c44 found three remaining bugs: an execution-module decoding fallback and two regressions in delimiter tracking, detailed inline. The four original reported examples are addressed in source.
I inspected all seven changed files and affected callers/tests, then checked every suggestion from a completed Greptile review of this exact comparison. An additional slash-context gap is already reported in the existing review, so I have not duplicated it.
This is source-only evidence. I have not run candidate tests, scanner reproductions, fresh-wheel checks or corpus scans. Behavioral verification and the full review remain incomplete, including issue #884's broader JavaScript/JSX/inline-HTML requirements.
|
Reproduction report for the five newest review items, run at head Two of the five items do not reproduce as described. Those corrections are called out per item, because a fix written from the review text alone would miss the real cause or break a passing case. Item 5 (escaped module specifier): reproduced, but only for the identity escapeExpected: Actual at head: owned. The guard is bypassed. Fails before any fix: yes.
Cause: Smallest regression test: def test_an_identity_escape_in_a_module_specifier_is_unowned():
source = 'import launch from "ex\\eca";\nlaunch(`${command.name}`);'
assert typescript_template_spans(source, lambda: None) is NoneItem 6 (nested calls and arrays): reproducedExpected: Actual at head: both return Fails before any fix: yes, both cases. Cause: at Smallest regression test: two asserts, Item 7 (parenthesised object in interpolation): reproducedExpected: Actual at head: Fails before any fix: yes. Cause: at Smallest regression test: one assert, Item 8 (unicode identifier escape): reproduced, but the named cause is incompleteExpected: Actual at head: owned. Fails before any fix: yes. Cause: One note on test shape: Smallest regression test: def test_an_identifier_escape_spelling_eval_is_unowned():
assert typescript_template_spans("const x = `\\u0065val(payload)`;", lambda: None) is NoneItem 9 (slash after braces): P1 claim reproduced; the stated control case is broken for a different reasonExpected: Actual at head: the P1 claim holds. The control case does not hold today, and not because of the slash. Consequence for the fix: narrowing Smallest regression tests, both required: def test_division_after_an_object_literal_is_unowned():
source = "const n = {} / eval(payload) / 2;\nconst label = `${item.name}`;"
assert typescript_template_spans(source, lambda: None) is None
def test_a_block_close_does_not_end_the_scan():
source = "if (ok) { run(); }\nconst r = /ab+c/g;\nconst label = `${item.name}`;"
assert typescript_template_spans(source, lambda: None) is not NoneOverlap with the older brace-balance findingThe older finding about top-level brace balance and crossed delimiters was left untouched, but the block-close defect above is the same code path. It should be re-reviewed jointly with item 9 rather than fixed twice, and a fix for one must run the other's case. Status
No code was changed and no thread was resolved while producing this. The next step is to agree the corrected cases, then rebase onto current |
9db61d0 to
7834047
Compare
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists. The shared call list also carries the JavaScript keyword "function", which is a declaration keyword in TypeScript and must not suppress ownership of a benign template.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
… names
The bounded TypeScript template reader lost track of which bracket ends a scope, because the ``)`` and ``]`` handlers tested the other bracket's open and close counts, which are always equal once the opener has been popped, so a bracket inside a call or an array literal could end the scan early; delimiters are now stored as matched opener and closer pairs and each closer checks the pair it actually ends, and the paren boundary bookkeeping that only existed to compensate for that is removed.
A slash after ``}`` was always read as a regex, so ``const n = {} / eval(p) / 2`` skipped the call; a brace now divides. An undecodable module specifier is treated as naming the execution module instead of falling back to the raw text, and identifiers are matched with escapes and decoded before classification, so ``\u0065val(payload)`` can no longer slip past the call check while a benign escaped name stays owned. Escapes inside a template are literal text, so that regression is in code context only, and JSX attribute templates stay on the conservative path.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
|
Reassessment of the seven open findings against head Method: each reviewer snippet was run verbatim against Four older findings
Three newer findings (threads marked outdated by the force update)
Note on 5, 6, 7: the committed regressions cover the scanner directly, and I confirmed the analyzer outcome by hand. They do not yet have a committed end-to-end analyzer test the way the older findings do. If that gap matters for review I can add one. Related coverage from this headThis is where most of the change landed, so the checks are broader:
Executed results
No code was changed for this reassessment and no commits were pushed. Implicit approval is not claimed; reviewers should confirm the fixes and close the threads themselves. |
… names
The bounded TypeScript template reader lost track of which bracket ends a scope, because the ``)`` and ``]`` handlers tested the other bracket's open and close counts, which are always equal once the opener has been popped, so a bracket inside a call or an array literal could end the scan early; delimiters are now stored as matched opener and closer pairs and each closer checks the pair it actually ends, and the paren boundary bookkeeping that only existed to compensate for that is removed.
A slash after ``}`` was always read as a regex, so ``const n = {} / eval(p) / 2`` skipped the call; a brace now divides. An undecodable module specifier is treated as naming the execution module instead of falling back to the raw text, and identifiers are matched with escapes and decoded before classification, so ``\u0065val(payload)`` can no longer slip past the call check while a benign escaped name stays owned. Escapes inside a template are literal text, so that regression is in code context only, and JSX attribute templates stay on the conservative path.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
7834047 to
c4b1abe
Compare
|
Reassessment of P1: escaped identifier escapes the execution guard
Existing controls:
P1: slash after a closing brace
Existing controls:
Reverting the brace rule makes the first two fail, so the change is load bearing rather than incidental. One note on the review textThe P1 description says Verification basisBoth review snippets were run by hand at No code, threads or review state were changed for this comment. |
c4b1abe to
a2c6aae
Compare
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists. The shared call list also carries the JavaScript keyword "function", which is a declaration keyword in TypeScript and must not suppress ownership of a benign template.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
… names
The bounded TypeScript template reader lost track of which bracket ends a scope, because the ``)`` and ``]`` handlers tested the other bracket's open and close counts, which are always equal once the opener has been popped, so a bracket inside a call or an array literal could end the scan early; delimiters are now stored as matched opener and closer pairs and each closer checks the pair it actually ends, and the paren boundary bookkeeping that only existed to compensate for that is removed.
A slash after ``}`` was always read as a regex, so ``const n = {} / eval(p) / 2`` skipped the call; a brace now divides. An undecodable module specifier is treated as naming the execution module instead of falling back to the raw text, and identifiers are matched with escapes and decoded before classification, so ``\u0065val(payload)`` can no longer slip past the call check while a benign escaped name stays owned. Escapes inside a template are literal text, so that regression is in code context only, and JSX attribute templates stay on the conservative path.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision: Changes Requested
Value and readiness
The TypeScript/TSX ownership path addresses #884's corrected everyday-template reproducer, and the reported prior findings are now repaired. A distinct remaining slash-context case can still hide an execution call from the ownership guard, so this revision is not ready to approve.
Material findings
[P1, required] Preserve property-name context when deciding whether / starts a regex, src/skillspector/typescript_tokens.py:218. This valid TypeScript divides by the result of an ordinary eval call:
const label = `${obj.return / eval(payload) / 2}`;The identifier branch at :581 overwrites the preceding . context and records return as a plain identifier. regex_allowed() then classifies it using _EXPRESSION_PREFIX_WORDS, although return here is a member name and its completed member expression is the left division operand. The first slash therefore enters skip_regex, which consumes / eval(payload) / without visiting execution_name('eval'). The remaining interpolation and source delimiters balance, so the enclosing dynamic template receives an owned span. _TypeScriptTemplateOwnership exposes that span through owns(), and static_patterns_tool_misuse.py:3767 skips the template's shell-command candidates. This drops the conservative completeness evidence that the execution guard is intended to preserve. Retain the token's member/keyword role, or fail closed when that role is uncertain, and add dotted-keyword division cases through both the span helper and ledger, with genuine keyword/regex controls. The new brace and postfix controls do not cover this context. This is a static lexical/caller trace, not a locally executed scanner-verdict claim.
All prior required examples were checked and are resolved rather than repeated: escaped module/computed members; exact lowercase function versus Function; loader context through (; postfix ++/--; unmatched/crossed delimiters; undecodable module fallback; call/array delimiter pairing and nested interpolation objects; escaped identifiers; and slash-after-object-brace handling. Their source guards and committed regression cases remain present.
PIC tradeoffs
The stated scope can remain TypeScript/TSX; JavaScript/JSX and inline HTML follow separate paths. Improving completeness must still retain conservative coverage when lexical ambiguity could hide execution. No new public-output policy decision is needed for the correction above.
Verification and gaps
Reviewed the complete actual diff, full new scanner and both test files, changed ownership/file-mapping callers and adjacent context, exact-head shared JavaScript execution-name list, #884 plus all corrected-trigger/impact followups, every prior review/reply/discussion and thread state, and all six successful hosted checks. Seven substantive external review threads remain unresolved despite the source corrections. Pre-fetch, post-fetch and post-review heads match the frozen inventory. Author runtime and mutation-test reports were read as supplied evidence, not independently reproduced. No contributor code, tests, imports, builds, hooks, scanner, regex probes or model calls were executed.
Reviewed head: a2c6aae98a36d82420cdefbf1fb3f753eeb2587f.
…regex
After a dot, ``return``, ``delete``, and ``typeof`` are property names, not expression prefixes, but the scanner classified the identifier by spelling alone, so ``${obj.return / eval(payload) / 2}`` read the middle slash as a regex opener, skipped the ``eval`` call, and claimed the template; the same hole existed after ``#`` for private names such as ``this.#return``. Track whether the last identifier followed ``.`` or ``#`` and let that member flag veto the keyword set before the slash decision. Add span-helper regressions for the three dotted keywords, the same cases through the inspection ledger, a private-name case, and controls that keep member division benign and a genuine keyword regex a regex; the new cases fail on the previous revision.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
|
Progress update at The latest P1 finding is fixed: after All seven review threads have current-head evidence and are resolved. The cited regressions pass here, including the analyzer-level outcomes for the three newer findings. CI on This is a progress update, not a self-approval. Maintainers should make the formal decision. |
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision: Changes Requested
Value and readiness
Everyday TypeScript templates benefit from ownership proof. The direct keyword-property correction is present. The proof still treats a related member call as a control statement, so the execution boundary needs another correction.
Material findings
Required — src/skillspector/typescript_tokens.py:454–470. The opening parenthesis records a control head solely from the identifier text, even after the new identifier branch has recorded that the name is a member. In a valid expression such as obj.if() / eval(input) / 2 inside a template interpolation, the first closing parenthesis sets last_closed_control to true. The slash then starts the regex-skip path at lines 229–230 and 441–450. It skips the execution-name check for the expression between the slashes and can grant ownership to the enclosing template. Use the recorded member role in the control-head decision, or withhold ownership when the role cannot be established. Add a member-call division regression through both the span helper and the completeness ledger, with a genuine control-head regex as the positive control.
The prior direct keyword-property case and the earlier delimiter, escaped-name and module safeguards remain covered by the current code and tests. No duplicate thread is needed.
PIC tradeoffs
None requiring a separate decision. The stated scope remains TypeScript template ownership with a conservative fallback.
Verification and gaps
Compared the complete current feature diff with the fully reviewed previous feature diff. Read the changed lexer state and test cases, the caller ownership path, current review threads and author reply. Six hosted checks succeeded. Findings follow the source control flow. No contributor code, tests or scanner were executed locally.
Reviewed head: b4c142b9ac156f6bd41fd1e0686e0746b3adbd41.
Add a bounded TypeScript template scanner used only when the whole file is available. Benign template literals no longer borrow shell backtick ownership, while tagged templates, shell execution surfaces, non-host shebangs, malformed delimiters, and ambiguous regex/division syntax stay on the conservative path. Map .tsx files to TypeScript so the same path applies to TSX sources. Refs NVIDIA#884 Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
…checks
The TypeScript template scanner compared raw source bytes against the shell execution module allowlist and the shell execution call names, so an escaped specifier such as "node:child_\x70rocess" or a computed member such as cp["exec"] slipped past the checks and the file was reported as fully inspected. Decode \xNN, \uNNNN, \u{...}, the single-character and line-continuation escapes without evaluation, return None on malformed, unsupported, or partially decoded strings so the conservative read and its partial status remain, and compare decoded module specifiers and computed or dotted member names against the existing lists. The shared call list also carries the JavaScript keyword "function", which is a declaration keyword in TypeScript and must not suppress ownership of a benign template.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
Four review findings let the bounded TypeScript template reader claim ownership of byte ranges it had not proven:
- The casefolded call-name comparison folded ``Function`` into the ``function`` declaration keyword, so ``new Function(`${code}`)()`` was owned. Only the exact lowercase keyword is a declaration now.
- The ``(`` handler overwrote the last token, so module specifiers in ``require(...)`` / ``import(...)`` were never compared with the execution-module list. The callee name now survives the paren.
- A collided ``+`` left ``+`` as the last token, so ``counter++ / eval(p) /`` read as a regex and skipped the ``eval`` guard. Postfix ``++``/``--`` are consumed as one token.
- Brace balance was only tracked in interpolation mode and delimiter nesting was never ordered, so an incomplete block or a crossed ``([)]`` still returned spans. A shared delimiter stack now rejects both.
Each finding has a regression that fails on the previous revision and passes here; the benign guards stay owned.
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
… names
The bounded TypeScript template reader lost track of which bracket ends a scope, because the ``)`` and ``]`` handlers tested the other bracket's open and close counts, which are always equal once the opener has been popped, so a bracket inside a call or an array literal could end the scan early; delimiters are now stored as matched opener and closer pairs and each closer checks the pair it actually ends, and the paren boundary bookkeeping that only existed to compensate for that is removed.
A slash after ``}`` was always read as a regex, so ``const n = {} / eval(p) / 2`` skipped the call; a brace now divides. An undecodable module specifier is treated as naming the execution module instead of falling back to the raw text, and identifiers are matched with escapes and decoded before classification, so ``\u0065val(payload)`` can no longer slip past the call check while a benign escaped name stays owned. Escapes inside a template are literal text, so that regression is in code context only, and JSX attribute templates stay on the conservative path.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
…regex
After a dot, ``return``, ``delete``, and ``typeof`` are property names, not expression prefixes, but the scanner classified the identifier by spelling alone, so ``${obj.return / eval(payload) / 2}`` read the middle slash as a regex opener, skipped the ``eval`` call, and claimed the template; the same hole existed after ``#`` for private names such as ``this.#return``. Track whether the last identifier followed ``.`` or ``#`` and let that member flag veto the keyword set before the slash decision. Add span-helper regressions for the three dotted keywords, the same cases through the inspection ledger, a private-name case, and controls that keep member division benign and a genuine keyword regex a regex; the new cases fail on the previous revision.
Refs NVIDIA#990
Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
A keyword after `. ` or `#` is a member name, so `obj.if()` is a call rather than a statement head. The opening paren still recorded a control head from the identifier text, and the matching close then let the following slash open a regex that skipped `eval(input)` and claimed the template. Include the recorded member role in the control-head decision. Add member-call division regressions through the span helper and the completeness ledger, plus a genuine `if (...)` regex control. Rebase onto current main also moved module matching into `javascript_callables` and switched this test to the native parser, so import the shared matcher and use a valid lone-surrogate escape in the undecodable-string case. Refs NVIDIA#884. Signed-off-by: Niladri Das <125604915+bniladridas@users.noreply.github.com>
b4c142b to
7ce5e95
Compare
|
Rebased onto |
Summary
Adds a bounded TypeScript template scanner and applies it only when the whole file is available. Benign
.tstemplates no longer reach the shell backtick parser, including the${format(n)}reproducer. Tagged templates, shell execution surfaces, non-host shebangs, malformed delimiters, and ambiguous regex syntax stay conservative.Maps
.tsxto TypeScript so the same ownership path applies to TSX sources. JSX-specific syntax still fails closed, including JSX attribute templates..jsxremains outside this change.The scanner reads delimiters as matched opener and closer pairs, so a closer only ends the scope it actually pairs with. The earlier revision compared cross-bracket counts that were always equal and also carried a paren boundary stack to compensate, which let a bracket inside a call or an array end the scan early. That bookkeeping is gone here.
A slash after
}divides rather than opening a regex. A brace ends an expression far more often than a statement, soconst n = {} / eval(p) / 2now reaches theevalguard instead of being read as a regex. Statement-position blocks that are followed by a regex still parse, and a nested block inside a template keeps later templates owned.Identifiers are matched with escape sequences and decoded before they are classified, so
\u0065val(payload),\x65val(payload)and\u{65}val(payload)are all theevalcall they spell, while a benign escaped name stays owned. An escape inside a template is literal text, so that case is out of scope for an ownership bypass and covered only in code context.An undecodable module specifier fails closed.
_decode_string_sourcerejects an unknown escape such as the identity escape inex\eca, so the raw spelling may still nameexeca; the reader now treats that as an execution module instead of falling back to a raw-string comparison.After
.or#, an identifier is a property name even when it spells a keyword.${obj.return / eval(payload) / 2}used to read the middle slash as a regex opener, skip theevalcall and claim the template;this.#returnhad the same hole. The scanner now records member context from the preceding token and vetoes the keyword set before the slash decision.Review follow-up
Expand the five review follow-ups
Review follow-up from
rng1995(changes requested onde9e5a1d): the shared call list also carries the JavaScript keywordfunction, which is a declaration keyword in TypeScript. It is excluded from the TypeScript comparison, so a generic function body no longer suppresses ownership of a benign template;test_declaration_keywords_do_not_read_as_execution_callsfails without that exclusion. String spellings are decoded before the guard compares them._decode_string_sourceresolves\xNN,\uNNNN,\u{...}, the single-character and line-continuation escapes without evaluation and returnsNoneon malformed, unsupported, or partially decoded input, so an undecodable specifier keeps the conservative read instead of establishing or clearing ownership. Decoded module specifiers afterfrom,import, andrequireare checked against the allowlist, and decoded computed and dotted members such ascp["exec"],cp[`exec`], andcp.execare checked against_SHELL_EXECUTION_CALLS.Review follow-up from
yashrajp22(four findings raised onb15673fd): four cases could still claim ownership of byte ranges the reader had not proven. Only the exact lowercasefunctionis a declaration keyword, soFunction, includingnew Function(${code})(), stays an execution surface. The callee name survives the(token, so the module specifier inrequire('execa')and dynamicimport(...)is compared with the execution-module list. Postfix++/--are consumed as a single token, socounter++ / eval(p)reads as division and does not skip theevalguard. Braces are balanced in host code as well as interpolation,${records its own brace, and unmatched or crossed(),[], and{}are rejected. A control head such asif (ok) {is still a statement, not a call.Review follow-up from
yashrajp22(three findings raised on9db61d04): an identity escape in a module specifier letex\ecamiss the module guard, the delimiter stack held only openers sodelimiters.count(")")was always zero and a bracket inside a call, such as${format(items[0])}, lost ownership, and the{handler pushed a paren boundary that only the non-interpolation close popped, so a valid nested object in an interpolation, such as${({name: item.name}).name}, was rejected. The first is fixed by failing closed on undecodable specifiers, the second and third by the matched-pair stack and the removal of the boundary bookkeeping. A mutation check confirms each change is load-bearing: reverting the brace rule, the module rule, the identifier decode, the paren pairing or the bracket pairing each fails at least one committed test.Review follow-up from
yashrajp22(P1 raised on 2026-11-13): the slash classifier lost the member role of a keyword after a dot, so${obj.return / eval(payload) / 2}skipped theevalguard. Member context after.and#is now preserved, with span-helper and inspection-ledger regressions forreturn,delete, andtypeof, a private-name case, and controls that keep a genuine keyword regex a regex. The dotted and private cases fail against the previous revision.Review follow-up from
rng1995(required finding raised onb4c142b9): the opening parenthesis recorded a control head from the identifier text alone, even when the identifier was a member. Inobj.if() / eval(input) / 2the first close then setlast_closed_control, so the next slash opened the regex-skip path and could grant ownership. The control-head decision now includes the recorded member role, so a keyword member call does not become a statement head. The new span-helper regression fails on the previous revision, the ledger case exercises the same path, and a genuineif (ok)regex remains a regex.Verification
Rebased onto upstream
mainat48e0f942(125 commits ahead of the previous merge base3c2c5d7a), so the branch is linear and six commits deep with no merge commit:3684151aadds the bounded scanner,c340e45cdecodes string escapes before the shell-surface checks,47627367closes the four older findings fromyashrajp22,e333c99dcloses the three newer ones,498ddfb8keeps keyword property names from opening a regex, and7ce5e954keeps member calls out of control heads. The merge base withupstream/mainis48e0f942. Conflicts wereCHANGELOG.md, where both branches added an entry at the same insertion point, and the analyzer import block; the resolution keeps both entries and both imports. The rebase also adapted the moved_is_shell_execution_moduleimport and kept the undecodable-benign-string test on a valid lone-surrogate escape for the native parser.As run in this checkout on
7ce5e954:pytest tests/nodes/analyzers/test_typescript_template_ownership.py tests/nodes/analyzers/test_typescript_template_ownership_delimiters.py tests/nodes/analyzers/test_host_language_shell_ownership.py tests/nodes/analyzers/test_javascript_template_ownership.py -q: 1148 passedpytest tests/nodes/analyzers -q: 16928 passed, 4 skipped, 5 deselected, 4 xfailedruff check src/ tests/: passedruff format --check src/ tests/: 428 files already formattedThe full non-integration run was started locally and reached 64% with no failures before a 20 minute command timeout; CI runs the complete suite. The previous full run on the pre-rebase tree passed with 24723 passed, 40 skipped, 261 deselected, 4 xfailed in 1041.65s.
The new regressions in
test_typescript_template_ownership_delimiters.pywere run with the relevant change reverted: reverting the brace rule fails the division-only and the sink regressions, reverting the module rule fails the identity-escape case, reverting the identifier decode fails the four escape spellings, and reverting either delimiter pairing fails the bracket cases. Each new case therefore fails without its fix. The benign companions stay owned on both revisions, which keeps the guards from overreaching. The member-context regressions were also run against the previous scanner: the seven dotted cases and the private-name case fail there, while the genuine keyword-regex control passes on both revisions. The new member-call span-helper regression returned spans on the previous scanner, while the genuineif (ok)regex control passed on both revisions.The
.tsand.tsxCLI repro (skillspector scan --no-llm --format json) on the pre-rebase headb4c142b9reportsanalysis_completeness.statusiscompleteandcoverage_percentis100.0, with no issues.CI on
7ce5e954is queued after the force-push.Scope
This is not a TypeScript parser or a shell-sink/dataflow model. It proves template literal ranges for complete TypeScript source and keeps the original conservative shell reading for anything it cannot prove. It does not model data flow through decoded names, so decoded comparisons apply to literal spellings only. JSX attribute templates are not owned and are not covered by a dedicated test; the conservative rejection stays until JSX syntax has its own coverage.
Refs #884