diff --git a/.changeset/block-argument-type.md b/.changeset/block-argument-type.md new file mode 100644 index 000000000..a92e859f4 --- /dev/null +++ b/.changeset/block-argument-type.md @@ -0,0 +1,6 @@ +--- +'@shopify/liquid-html-parser': minor +'@shopify/theme-language-server-common': patch +--- + +Type `{% block %}` and `{% section %}` arguments as one `BlockArgument`, whose value is a scalar or a single-level array literal, instead of a `LiquidNamedArgument | BlockArrayArgument` union. Completions narrow on the argument's value. diff --git a/.changeset/html-comment-children.md b/.changeset/html-comment-children.md new file mode 100644 index 000000000..e9ebe2782 --- /dev/null +++ b/.changeset/html-comment-children.md @@ -0,0 +1,25 @@ +--- +'@shopify/liquid-html-parser': minor +'@shopify/prettier-plugin-liquid': patch +'@shopify/theme-check-common': patch +'@shopify/theme-language-server-common': patch +--- + +Parse the Liquid inside HTML comments. Liquid renders it, and its tags can open, branch and close blocks, so `{% if a %}` no longer fails with an unclosed `if`. + +- `HtmlComment` has `children` (text and Liquid) and `blockStartPosition`/`blockEndPosition` for its delimiters. +- A comment that crosses a Liquid block boundary is split there, and each part has an empty delimiter range on the split side. +- A comment that never closes ends at the end of the file, as in a browser. +- An end tag in a comment that closes no block is a parse error, as in Liquid. +- The `opaqueHtmlComments` option keeps comment bodies opaque. + +**Behavior changes** + +- Code that walks `children` now visits the Liquid and text inside HTML comments. +- `

x

` is one comment whose `if` holds `

x

` as text, not two comments around a `

` element. +- `` with no open `if` no longer parses. An unclosed `` now parse. +- Code that creates `HtmlComment` nodes must set `children`, `blockStartPosition` and `blockEndPosition`. + +Pass `opaqueHtmlComments: true` to get the previous tree for these templates. + +Checks now see the Liquid in HTML comments, and completions work inside them. Prettier prints an unclosed comment as written, and can move whitespace and line breaks inside a comment that holds a Liquid block or is split by one. diff --git a/.changeset/html-comment-mode.md b/.changeset/html-comment-mode.md new file mode 100644 index 000000000..45689b679 --- /dev/null +++ b/.changeset/html-comment-mode.md @@ -0,0 +1,12 @@ +--- +'@shopify/liquid-html-parser': minor +--- + +Tokenize HTML comments as a browser does. In a comment only Liquid and the comment's close count, so a ``. A comment closes on `-->` or `--!>`, and `` and `` are empty comments. Outside a comment, `-->` is text. The bodies of Liquid raw tags like `{% comment %}` are opaque to the tokenizer too, so a `

shown
-->` is an empty comment, a `
` element and the text `-->`. It used to be one comment. +- `

b

` closes the comment at `--!>` instead of failing to parse. + +`opaqueHtmlComments` doesn't change these. diff --git a/.changeset/linear-text-trim.md b/.changeset/linear-text-trim.md new file mode 100644 index 000000000..95738e0b8 --- /dev/null +++ b/.changeset/linear-text-trim.md @@ -0,0 +1,5 @@ +--- +'@shopify/liquid-html-parser': patch +--- + +Trim whitespace around Liquid tags with `trimStart()`/`trimEnd()` instead of regular expressions. The trailing-whitespace regex was quadratic in the length of a whitespace run and was the single most expensive step of parsing a typical theme. diff --git a/.changeset/markup-tokenizer-fast-checks.md b/.changeset/markup-tokenizer-fast-checks.md new file mode 100644 index 000000000..1bbfed464 --- /dev/null +++ b/.changeset/markup-tokenizer-fast-checks.md @@ -0,0 +1,5 @@ +--- +'@shopify/liquid-html-parser': patch +--- + +Check character codes before running regular expressions or `startsWith` in the Liquid expression tokenizer and when looking for `{{`/`{%`. Tokens and ASTs are unchanged; parsing is 5–7% faster. diff --git a/.changeset/pair-dangling-close-tags.md b/.changeset/pair-dangling-close-tags.md new file mode 100644 index 000000000..f854f4618 --- /dev/null +++ b/.changeset/pair-dangling-close-tags.md @@ -0,0 +1,5 @@ +--- +'@shopify/liquid-html-parser': minor +--- + +Add a `pairDanglingCloseTags` option, off by default. Themes often open an element in each branch of a conditional and close it once after it (`{% if a %}
{% else %}
{% endif %}…
`). With the option, such a close tag becomes an `HtmlDanglingMarkerClose` when an earlier `if`, `unless` or `case` in the same scope leaves an element of that name open on every path, instead of a parse error. It pairs only with the innermost such element. diff --git a/.changeset/prototype-names-not-keywords.md b/.changeset/prototype-names-not-keywords.md new file mode 100644 index 000000000..db605cabd --- /dev/null +++ b/.changeset/prototype-names-not-keywords.md @@ -0,0 +1,5 @@ +--- +'@shopify/liquid-html-parser': patch +--- + +Parse names inherited from `Object.prototype` like any other name. `{{ constructor }}` and `{{ toString }}` were parsed as Liquid literals whose value was a JavaScript function, and `{% constructor %}` or `{% __proto__ %}` made the parser throw instead of producing an unknown tag. Recognizing literal keywords with a `switch` is also slightly faster. diff --git a/.changeset/raw-tag-body-rules.md b/.changeset/raw-tag-body-rules.md new file mode 100644 index 000000000..49b77003f --- /dev/null +++ b/.changeset/raw-tag-body-rules.md @@ -0,0 +1,12 @@ +--- +'@shopify/liquid-html-parser': patch +--- + +End each raw tag's body by the rule of the Liquid class that parses it, in templates and in `{% liquid %}`: + +- `{% comment %}` nests, skips the bodies of raw tags inside it, and closes on any tag named `endcomment`, so `{% endcomment xyz %}` closes it. +- `{% raw %}`, `{% javascript %}`, `{% schema %}` and `{% stylesheet %}` close at their first end tag, extra markup included, so `{% endraw xyz %}` closes `{% raw %}`. +- `{% doc %}` reads like `{% raw %}` but rejects a nested `{% doc %}`. +- `{% style %}` is a block, so it nests. + +Templates that Liquid reports as never closed now fail to parse too, such as a comment whose body Liquid's tokenizer runs past the `{% endcomment %}`. diff --git a/.changeset/script-style-raw-text.md b/.changeset/script-style-raw-text.md new file mode 100644 index 000000000..f79c97b31 --- /dev/null +++ b/.changeset/script-style-raw-text.md @@ -0,0 +1,9 @@ +--- +'@shopify/liquid-html-parser': minor +--- + +Read `c` now ends the script at the first `` and reports the second as an unmatched close tag. `` still pairs nested `` elements. + +**Behavior changes** + +- A ``. `` and `` now fail to parse in `strict` mode with "Attempting to close HtmlElement 'script' before it was opened". diff --git a/.changeset/tokenizer-text-fast-path.md b/.changeset/tokenizer-text-fast-path.md new file mode 100644 index 000000000..45a3e8dcd --- /dev/null +++ b/.changeset/tokenizer-text-fast-path.md @@ -0,0 +1,7 @@ +--- +'@shopify/liquid-html-parser': patch +--- + +Tokenize runs of plain text in one step instead of one character at a time. Most characters can't start a token, so the tokenizer now jumps to the next one that can. + +Tokens and ASTs are unchanged. On Dawn, Horizon and the base theme, `tokenize` is about 6× faster and `toLiquidHtmlAST`/`toLiquidAST` about 2× faster. diff --git a/packages/liquid-html-parser/specs/architecture.md b/packages/liquid-html-parser/specs/architecture.md index 428c1d92e..b68d85af6 100644 --- a/packages/liquid-html-parser/specs/architecture.md +++ b/packages/liquid-html-parser/specs/architecture.md @@ -146,7 +146,7 @@ The parser calls these utilities while building the AST. | Module | Owns | Does NOT do | | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `document/tokenizer.ts` | Modal document tokenizer. Default mode: scan for `{%`, `%}`, `{{`, `}}`, `<`, `>`, `/>`, ``, ``, `/>`, ` HtmlSelfClose, // /> HtmlCommentOpen, // + HtmlCommentClose, // --> or --!>, or > / -> right after ` or `--!>`, or `>` or `->` right after `` and ``). Outside a comment, `-->` is `Text`. Liquid inside a comment is still tokenized, because it still renders. Outside SVG, a `

a

', options); + expectPath(ast, 'children.1.type').to.eql('HtmlComment'); + expectPath(ast, 'children.1.children').to.eql([]); + expectPath(ast, 'children.1.body').to.eql('{% endif %} {% raw %} {{ b }}'); + }); + + it('should reject an unclosed HTML comment', () => { + expect(() => toLiquidHtmlAST('` */ +/** + * Represents ``. + * + * Liquid treats `` as plain text, so the Liquid inside a comment + * still renders and can close a block opened before the comment. + * + * This node describes the source, not the rendered page. The parser knows where + * each delimiter is written, not which `-->` closes which `` sits inside a block it opened ends with + * that block. Liquid outputs every delimiter as written, so only the rendered + * output shows which comments are open. + */ export interface HtmlComment extends ASTNode { + /** The source between the delimiters, trimmed. */ body: string; + /** The text and Liquid between the delimiters. */ + children: (LiquidNode | TextNode)[]; + /** The range covered by this node's ``, empty when the node ends at a block boundary. */ + blockEndPosition: Position; } export interface HtmlNodeBase extends ASTNode { @@ -932,6 +959,27 @@ export interface ASTBuildOptions { /** Optional environment for tag registration. Uses Environment.default() if omitted. */ environment?: Environment; + + /** + * Parse a close tag as an `HtmlDanglingMarkerClose` when it names the + * innermost element that earlier `if` / `unless` / `case` tags in the same + * scope left open on every path, instead of matching it against the enclosing + * element: + * + * {% if a %}
{% else %}
{% endif %}...
+ * + * Without it such a template throws. Liquid treats the markup as text, so + * Shopify renders it, and a renderer can pair the marker with the open + * element. Off by default: it accepts templates the strict grammar rejects. + */ + pairDanglingCloseTags?: boolean; + + /** + * Keep each HTML comment's body as opaque text instead of parsing the Liquid + * inside it. For HTML that is data rather than a template, such as a rendered + * setting value or a translation, where `{% ... %}` inside a comment is not a tag. + */ + opaqueHtmlComments?: boolean; } export function isBranchedTag(node: LiquidHtmlNode) { @@ -946,7 +994,9 @@ export function toLiquidAST( }, ) { const env = options.environment ?? Environment.default(); - const tokens = tokenize(source); + const tokens = tokenize(source, { + rawBodyRule: rawBodyRuleLookup((name) => env.tagForName(name)), + }); const parser = new DocumentParser(tokens, source, env, false, options.allowUnclosedDocumentNode); return parser.parseDocument(); } @@ -959,8 +1009,18 @@ export function toLiquidHtmlAST( }, ): DocumentNode { const env = options.environment ?? Environment.default(); - const tokens = tokenize(source); - const parser = new DocumentParser(tokens, source, env, true, options.allowUnclosedDocumentNode); + const tokens = tokenize(source, { + rawBodyRule: rawBodyRuleLookup((name) => env.tagForName(name)), + }); + const parser = new DocumentParser( + tokens, + source, + env, + true, + options.allowUnclosedDocumentNode, + options.opaqueHtmlComments, + ); + parser.pairDanglingCloseTags = options.pairDanglingCloseTags ?? false; return parser.parseDocument(); } diff --git a/packages/liquid-html-parser/src/document/base.ts b/packages/liquid-html-parser/src/document/base.ts index 57a8d6acc..62a5fb469 100644 --- a/packages/liquid-html-parser/src/document/base.ts +++ b/packages/liquid-html-parser/src/document/base.ts @@ -1,6 +1,7 @@ -import { TokenType, createTokenizer } from './tokenizer'; -import type { Token, Tokenizer } from './tokenizer'; +import { TokenType, createTokenizer, tokenize } from './tokenizer'; +import type { Token, TokenizeOptions, Tokenizer } from './tokenizer'; import { LiquidHTMLASTParsingError } from '../errors'; +import type { RawBodyRuleFor } from './raw-body'; export class ParserBase { protected tokens: Token[]; @@ -16,6 +17,11 @@ export class ParserBase { private scanned = 0; private openQuote: Token | undefined; private openTag = false; + /** + * The tag registry's raw-tag body rules, passed to every re-tokenize so raw + * bodies stay opaque there too (see `TokenizeOptions.rawBodyRule`). + */ + protected rawBodyRule: RawBodyRuleFor | undefined; constructor(tokens: Token[], source: string) { this.tokens = tokens; this.source = source; @@ -208,6 +214,14 @@ export class ParserBase { const before = this.countTokensWhile((token) => token.end <= sourceOffset); const straddles = this.hasToken(before) && this.tokens[before].start < sourceOffset; + if (straddles && this.rawTextBound !== null && sourceOffset < this.rawTextBound) { + const bound = this.rawTextBound; + const rest = this.countTokensWhile((token) => token.start < bound); + this.replaceTokens(before, rest, this.tokenizeSlice(sourceOffset, bound, { rawText: true })); + this.p = before; + return; + } + if (straddles) { this.scanHtmlContext(before); // If the boundary lies inside a quoted HTML attribute value (the raw tag @@ -228,6 +242,7 @@ export class ParserBase { this.tokens.length = before; this.tokenizer ??= createTokenizer(this.source, this.tokens); this.tokenizer.restart(sourceOffset, { + rawBodyRule: this.rawBodyRule, skipFrontmatter: true, insideQuotedAttribute, insideHtmlTag, @@ -243,4 +258,74 @@ export class ParserBase { // never skipped past, keeping `this.p` in bounds.) this.seekToSourceOffset(sourceOffset); } + + /** + * Re-tokenize the ` ')).toContainEqual([TokenType.HtmlCommentOpen, '')).toContainEqual([ + TokenType.HtmlCommentOpen, + ' outside a comment as text', () => { + expect(texts('a --> b')).toEqual([[TokenType.Text, 'a --> b']]); + }); + + it("leaves an unclosed comment's body as text to the end", () => { + expect(texts('` is text. + const source = 'a-b-->cf'; + expect(tokens(source)).toMatchObject([ + { type: TokenType.Text, start: 0, end: 7 }, + { type: TokenType.HtmlCommentOpen, start: 7, end: 11 }, + { type: TokenType.Text, start: 11, end: 14 }, + { type: TokenType.HtmlCommentClose, start: 14, end: 17 }, + { type: TokenType.Text, start: 17, end: 18 }, + ]); + assertTokenInvariants(source); + }); + }); + + describe('text fast path stops at every token start mid-text', () => { + // No character in `pad` can start a token in any mode, so the fast path + // skips the whole pad and must stop exactly where the token begins. + const pad = 'abc 1 '; + const quotePairs = [ + ['"', '"'], + ["'", "'"], + ['\u201c', '\u201d'], + ['\u201d', '\u201c'], + ['\u2018', '\u2019'], + ['\u2019', '\u2018'], + ]; + const cases: Array<[string, string, TokenizeOptions, string[]]> = [ + [ + 'Default', + '', + {}, + ['{{', '{{-', '{%', '{%-', '', '--!>']], + ['RawText script', '', '']], + [ + 'HtmlTag', + '', + { insideHtmlTag: true }, + [ + '{{', + '{{-', + '{%', + '{%-', + '/>', + '>', + '=', + '"', + "'", + '\u201c', + '\u201d', + '\u2018', + '\u2019', + ], + ], + ...quotePairs.map(([open, close]): [string, string, TokenizeOptions, string[]] => [ + `QuotedValue ${open}`, + '', + { insideQuotedAttribute: open }, + [...new Set(['{{', '{%', open, close])], + ]), + ['LiquidTag', '{% ', {}, ['%}', '-%}']], + ['LiquidVariableOutput', '{{ ', {}, ['}}', '-}}']], + ]; + + for (const [mode, prefix, options, starts] of cases) { + it.each(starts)(`${mode}: %j`, (start) => { + const offset = prefix.length + pad.length; + const result = tokenize(prefix + pad + start + pad, options); + expect(result.some((t) => t.type !== TokenType.Text && t.start === offset)).toBe(true); + }); + } + }); + + describe('matches tokenizeWithoutFastPath', () => { + // Every string of up to 4 of these characters, in every entry state. The + // set covers each character a mode's token can start with, plus near + // misses (`!`, `\u201a`) and plain text. + const alphabet = [...'{}%-<>/=!"\'\u201c\u201d\u2018\u2019\u201a a\n']; + let sources = ['']; + for (let length = 1, previous = ['']; length <= 4; length++) { + previous = previous.flatMap((s) => alphabet.map((c) => s + c)); + sources = sources.concat(previous); + } + + const entryStates: Array<[string, string, TokenizeOptions]> = [ + ['document start', '', {}], + ['skipFrontmatter', '', { skipFrontmatter: true }], + ['insideHtmlTag', '', { insideHtmlTag: true }], + ['rawText', '', { rawText: true }], + ...['"', "'", '\u201c', '\u201d', '\u2018', '\u2019'].map( + (quote): [string, string, TokenizeOptions] => [ + `insideQuotedAttribute ${quote}`, + '', + { insideQuotedAttribute: quote }, + ], + ), + ['inside a comment', '', + '', + '', + '', + '', + '', + '', + '` and ``, which close only right after `` tokenize as they did before comment mode. + // Nested `` opens it. + let pendingElement: 'svg' | 'script' | 'style' | undefined; + // The ``, whitespace allowed before `>`. + function closesRawText(at: number): boolean { + if (rawTextTag === '') return false; + const name = source.slice(at + 2, at + 2 + rawTextTag.length); + if (name.toLowerCase() !== rawTextTag) return false; + let i = at + 2 + rawTextTag.length; + while (i < source.length && /\s/.test(source[i]!)) i++; + return source[i] === '>'; + } + + // At the `>` of an open tag: enter the element it named, if it matters. An + // `` starts an SVG body. A `

a

'; + const ast = toTolerantLiquidHtmlAST(source); + + expect(ast.children.map((child) => child.type)).toEqual([ + NodeTypes.HtmlComment, + NodeTypes.LiquidErrorNode, + NodeTypes.HtmlComment, + NodeTypes.HtmlElement, + ]); + expect(source.slice(ast.children[2].position.start, ast.children[2].position.end)).toBe('-->'); + }); }); describe('findErrorNodeAtOffset', () => { diff --git a/packages/liquid-html-parser/src/tolerant.ts b/packages/liquid-html-parser/src/tolerant.ts index 27de69e64..885e803bc 100644 --- a/packages/liquid-html-parser/src/tolerant.ts +++ b/packages/liquid-html-parser/src/tolerant.ts @@ -14,6 +14,7 @@ import type { ASTBuildOptions, DocumentNode, LiquidErrorNode, LiquidHtmlNode } f import { walk } from './ast'; import { TolerantDocumentParser } from './document/tolerant-parser'; import { tokenize } from './document/tokenizer'; +import { rawBodyRuleLookup } from './document/raw-body'; import { Environment } from './environment'; import { NodeTypes } from './types'; @@ -30,7 +31,9 @@ export function toTolerantLiquidAST( }, ): DocumentNode { const env = options.environment ?? Environment.default(); - const tokens = tokenize(source); + const tokens = tokenize(source, { + rawBodyRule: rawBodyRuleLookup((name) => env.tagForName(name)), + }); const parser = new TolerantDocumentParser( tokens, source, @@ -54,13 +57,16 @@ export function toTolerantLiquidHtmlAST( }, ): DocumentNode { const env = options.environment ?? Environment.default(); - const tokens = tokenize(source); + const tokens = tokenize(source, { + rawBodyRule: rawBodyRuleLookup((name) => env.tagForName(name)), + }); const parser = new TolerantDocumentParser( tokens, source, env, true, options.allowUnclosedDocumentNode, + options.opaqueHtmlComments, ); return parser.parseDocument(); } diff --git a/packages/prettier-plugin-liquid/src/printer/printer-liquid-html.ts b/packages/prettier-plugin-liquid/src/printer/printer-liquid-html.ts index 1eabbaa38..ee7c83b59 100644 --- a/packages/prettier-plugin-liquid/src/printer/printer-liquid-html.ts +++ b/packages/prettier-plugin-liquid/src/printer/printer-liquid-html.ts @@ -253,7 +253,7 @@ function printNode( } case NodeTypes.RawMarkup: { - if (node.parentNode?.name === 'doc') { + if (node.parentNode?.type === NodeTypes.LiquidRawTag && node.parentNode.name === 'doc') { return printLiquidDoc(path as AstPath, options, print, args); } @@ -332,6 +332,18 @@ function printNode( } case NodeTypes.HtmlComment: { + const { blockStartPosition: open, blockEndPosition: close } = node; + // Part of a comment split at a Liquid block boundary, or one that never + // closes: adding the missing delimiter would change the output. On a side + // without one, the next parse reads the line break printed there as part + // of this node, so trim it to keep formatting idempotent. + if (open.start === open.end || close.start === close.end) { + let text = node.source.slice(node.position.start, node.position.end); + if (open.start === open.end) text = text.trimStart(); + if (close.start === close.end) text = text.trimEnd(); + return text; + } + const conditionalComment = getConditionalComment( node.source.slice(node.position.start, node.position.end), ); diff --git a/packages/prettier-plugin-liquid/src/test/html-comment/fixed.liquid b/packages/prettier-plugin-liquid/src/test/html-comment/fixed.liquid index c69c34c87..89db02420 100644 --- a/packages/prettier-plugin-liquid/src/test/html-comment/fixed.liquid +++ b/packages/prettier-plugin-liquid/src/test/html-comment/fixed.liquid @@ -69,3 +69,18 @@ It should support and indend the bodies of all IE comment variations + +It should print the parts of a comment split at a Liquid block boundary as written +{% if a %} + +{% endif %} + +It should print a comment as written when a block opened inside it holds its close + a {% endif %} +c + +It should print a comment that never closes as written +

a

+ + +It should print the parts of a comment split at a Liquid block boundary as written +{% if a %}{% endif %} + +It should print a comment as written when a block opened inside it holds its close + a {% endif %}c + +It should print a comment that never closes as written +

a

`, ``, `

" + docset: default + - name: should report an end tag inside an HTML comment that closes no block + files: + file.liquid: "" + docset: default + offenses: + - file: file.liquid + message: "Attempting to close LiquidTag 'if' before it was opened without a matching 'if'" + start: [0, 5] + highlight: "{% endif %}" diff --git a/specs/checks/liquid-syntax-error.yml b/specs/checks/liquid-syntax-error.yml index 7685f9c3a..0257d29b7 100644 --- a/specs/checks/liquid-syntax-error.yml +++ b/specs/checks/liquid-syntax-error.yml @@ -1037,8 +1037,8 @@ cases: offenses: - file: snippets/test.liquid message: "Attempting to end parsing before LiquidRawTag 'doc' was closed" - start: [0, 0] - highlight: "{% doc %}" + start: [0, 32] + highlight: "{% doc %" - name: accepts later balanced doc blocks after accepted doc closing markup files: snippets/test.liquid: "{% doc %}content{% enddoc foo %}{% doc %}later{% enddoc %}" @@ -1090,8 +1090,8 @@ cases: offenses: - file: snippets/test.liquid message: "Attempting to end parsing before LiquidRawTag 'raw' was closed" - start: [0, 0] - highlight: "{% raw %}" + start: [0, 26] + highlight: "{% raw %" - name: accepts later balanced raw blocks after accepted raw closing markup files: snippets/test.liquid: "{% raw %}c{% endraw foo %}{% raw %}later{% endraw %}" @@ -1278,17 +1278,17 @@ cases: snippets/test.liquid: "{% comment %}this is {{{ not }}} valid{{ liquid{% endcomment %}" offenses: - file: snippets/test.liquid - message: "Liquid syntax error: 'comment' tag was never closed" - start: [0, 13] - highlight: this is {{{ not }}} valid{{ liquid + message: "Attempting to end parsing before LiquidRawTag 'comment' was closed" + start: [0, 0] + highlight: "{% comment %}" - name: "reports LiquidSyntaxError for Ruby-stricter comment raw bodies: {% comment %}this is {% if liquid{% endcomment %}" files: snippets/test.liquid: "{% comment %}this is {% if liquid{% endcomment %}" offenses: - file: snippets/test.liquid - message: "Liquid syntax error: 'comment' tag was never closed" - start: [0, 13] - highlight: this is {% if liquid + message: "Attempting to end parsing before LiquidRawTag 'comment' was closed" + start: [0, 0] + highlight: "{% comment %}" - name: produces no diagnostics for inert closed Liquid tags inside comment bodies files: snippets/test.liquid: "{% comment %}{% if %}{% endcomment %}" diff --git a/specs/checks/undefined-object.yml b/specs/checks/undefined-object.yml index 4122443ec..480c67271 100644 --- a/specs/checks/undefined-object.yml +++ b/specs/checks/undefined-object.yml @@ -756,3 +756,12 @@ cases: } {% endschema %} docset: default + - name: reports an undefined object inside an HTML comment, which Liquid renders + files: + file.liquid: "" + docset: default + offenses: + - file: file.liquid + message: "Unknown object 'my_var' used." + start: [0, 8] + highlight: my_var diff --git a/specs/parser/html.yml b/specs/parser/html.yml index 7df09c1f6..817f73af4 100644 --- a/specs/parser/html.yml +++ b/specs/parser/html.yml @@ -2558,12 +2558,37 @@ cases: children: - type: HtmlComment body: hello {{ product.name }} + blockStartPosition: { start: [0, 0], text: } position: start: [0, 0] text: |- + children: + - type: TextNode + value: hello + position: { start: [1, 2], text: hello } + - type: LiquidVariableOutput + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [1, 10], text: " product.name " } + position: { start: [1, 8], text: "{{ product.name }}" } + markup: + type: LiquidVariable + rawSource: product.name + position: { start: [1, 11], text: "product.name " } + expression: + type: VariableLookup + name: product + position: { start: [1, 11], text: product.name } + lookups: + - type: String + value: name + position: { start: [1, 19], text: name } + lookupModes: [bareword] + filters: [] - name: should parse script tags as raw source: |- + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: for (var i=0;iif(a + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: '' } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: '' } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: if(aif (i + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: + start: [0, 0] + text: |- + + attributes: [] + body: + type: RawMarkup + kind: javascript + value: | + if (iif (a + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: if (avar r = / + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: var r = /// show if count + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: + start: [0, 0] + text: |- + + attributes: [] + body: + type: RawMarkup + kind: javascript + value: |- + // show if count for(var e=0;e + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: '' } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: '' } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: for(var e=0;e/* " + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlRawNode + name: style + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: css + value: "/* for(i=0;i + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlRawNode + name: SCRIPT + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: for(i=0;i + source: |- + + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: + position: + start: [0, 0] + text: |- + + attributes: [] + body: + type: RawMarkup + kind: javascript + value: for(i=0;iif (i<{{ max }}) {} + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: if (i<{{ max }}) {} + position: { start: [0, 8], text: "if (i<{{ max }}) {}" } + nodes: + - type: TextNode + value: if (i< + position: { start: [0, 8], text: if (i< } + - type: LiquidVariableOutput + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [0, 16], text: " max " } + position: { start: [0, 14], text: "{{ max }}" } + markup: + type: LiquidVariable + rawSource: max + position: { start: [0, 17], text: "max " } + expression: + type: VariableLookup + name: max + position: { start: [0, 17], text: max } + lookups: [] + lookupModes: [] + filters: [] + - type: TextNode + value: ) {} + position: { start: [0, 23], text: ") {}" } + - name: parses a Liquid block after a < comparison in a script body + source: + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: for(i=0;iif(a

after
+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "
after
" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: if(a } + blockEndPosition: { start: [0, 36], text:
} + position: { start: [0, 26], text:
after
} + name: + - type: TextNode + value: div + position: { start: [0, 27], text: div } + attributes: [] + children: + - type: TextNode + value: after + position: { start: [0, 31], text: after } + - name: closes a script body at its own close tag rather than a later script's + source: |- + +
hi
+ + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + +
hi
+ + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: if (a } + blockEndPosition: { start: [1, 7], text:
} + position: { start: [1, 0], text:
hi
} + name: + - type: TextNode + value: div + position: { start: [1, 1], text: div } + attributes: [] + children: + - type: TextNode + value: hi + position: { start: [1, 5], text: hi } + - type: HtmlRawNode + name: script + blockStartPosition: { start: [2, 0], text: } + position: { start: [2, 0], text: } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: x() + position: { start: [2, 8], text: x() } + nodes: + - type: TextNode + value: x() + position: { start: [2, 8], text: x() } + - name: does not close a script body on a close tag inside a drop + source: ' | json }}; + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "' | json }};" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "' | json }};" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: for(i=0;i' | json }}; + position: { start: [0, 8], text: "for(i=0;i' | json }};" } + nodes: + - type: TextNode + value: for(i=0;i' | json " } + position: { start: [0, 35], text: "{{ '' | json }}" } + markup: + type: LiquidVariable + rawSource: "'' | json" + position: { start: [0, 38], text: "'' | json " } + expression: + type: String + single: true + value: + position: { start: [0, 38], text: "''" } + filters: + - type: LiquidFilter + name: json + position: { start: [0, 49], text: " | json" } + args: [] + - type: TextNode + value: ; + position: { start: [0, 59], text: ; } + - name: does not close a script body on a close tag inside a comment + source: "{% endcomment %}" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% endcomment %}" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "{% endcomment %}" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: "if (i{% endcomment %}" + position: { start: [0, 8], text: "if (i{% endcomment %}" } + nodes: + - type: TextNode + value: if (i{% endcomment %}" } + body: + type: RawMarkup + kind: text + value: "old: " + position: { start: [0, 32], text: "old: " } + nodes: + - type: TextNode + value: "old: " + position: { start: [0, 32], text: "old: " } + - name: does not close a script body without a < comparison on a close tag inside a comment + source: "{% endcomment %}" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% endcomment %}" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "{% endcomment %}" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: "x(){% comment %}old: {% endcomment %}" + position: { start: [0, 8], text: "x(){% comment %}old: {% endcomment %}" } + nodes: + - type: TextNode + value: x() + position: { start: [0, 8], text: x() } + - type: LiquidRawTag + name: comment + markup: "" + markupPosition: { start: [0, 22], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 11], text: "{% comment %}" } + blockEndPosition: { start: [0, 38], text: "{% endcomment %}" } + position: { start: [0, 11], text: "{% comment %}old: {% endcomment %}" } + body: + type: RawMarkup + kind: text + value: "old: " + position: { start: [0, 24], text: "old: " } + nodes: + - type: TextNode + value: "old: " + position: { start: [0, 24], text: "old: " } + - name: does not close a script body on a close tag inside a doc + source: {% enddoc %} + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% enddoc %}" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "{% enddoc %}" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: if(i{% enddoc %} + position: { start: [0, 8], text: "if(i{% enddoc %}" } + nodes: + - type: TextNode + value: if(i{% enddoc %}" } + body: + type: RawMarkup + kind: text + value: + position: { start: [0, 26], text: } + nodes: + - type: LiquidDocDescriptionNode + name: description + isImplicit: true + isInline: true + position: { start: [0, 26], text: } + content: + type: TextNode + value: + position: { start: [0, 26], text: } + - name: does not close a script body on a close tag inside a raw + source: {% endraw %} + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% endraw %}" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "{% endraw %}" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: if(i{% endraw %} + position: { start: [0, 8], text: "if(i{% endraw %}" } + nodes: + - type: TextNode + value: if(i{% endraw %}" } + body: + type: RawMarkup + kind: text + value: + position: { start: [0, 26], text: } + nodes: + - type: TextNode + value: + position: { start: [0, 26], text: } + - name: does not close a style body on a close tag inside a comment + source: {% endcomment %} + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% endcomment %}" } + children: + - type: HtmlRawNode + name: style + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "{% endcomment %}" } + attributes: [] + body: + type: RawMarkup + kind: text + value: /* {% endcomment %} + position: { start: [0, 7], text: "/* {% endcomment %}" } + nodes: + - type: TextNode + value: /* {% endcomment %}" } + body: + type: RawMarkup + kind: text + value: + position: { start: [0, 34], text: } + nodes: + - type: TextNode + value: + position: { start: [0, 34], text: } + - name: closes a script body after a raw {{ in a string, and parses the markup after it + source:

{{ x }}

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: '

{{ x }}

' } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: '' } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: var o="{% raw %}{{{% endraw %}"; + position: { start: [0, 8], text: 'var o="{% raw %}{{{% endraw %}";' } + nodes: + - type: TextNode + value: var o=" + position: { start: [0, 8], text: var o=" } + - type: LiquidRawTag + name: raw + markup: "" + markupPosition: { start: [0, 22], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 15], text: "{% raw %}" } + blockEndPosition: { start: [0, 26], text: "{% endraw %}" } + position: { start: [0, 15], text: "{% raw %}{{{% endraw %}" } + body: + type: RawMarkup + kind: text + value: "{{" + position: { start: [0, 24], text: "{{" } + nodes: + - type: TextNode + value: "{{" + position: { start: [0, 24], text: "{{" } + - type: TextNode + value: '";' + position: { start: [0, 38], text: '";' } + - type: HtmlElement + blockStartPosition: { start: [0, 49], text:

} + blockEndPosition: { start: [0, 59], text:

} + position: { start: [0, 49], text: "

{{ x }}

" } + name: + - type: TextNode + value: p + position: { start: [0, 50], text: p } + attributes: [] + children: + - type: LiquidVariableOutput + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [0, 54], text: " x " } + position: { start: [0, 52], text: "{{ x }}" } + markup: + type: LiquidVariable + rawSource: x + position: { start: [0, 55], text: "x " } + expression: + type: VariableLookup + name: x + position: { start: [0, 55], text: x } + lookups: [] + lookupModes: [] + filters: [] + - name: closes a script body after a raw {{ before a < comparison + source:

after

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "

after

" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: "{% raw %}{{{% endraw %}if(a } + blockEndPosition: { start: [0, 57], text:

} + position: { start: [0, 49], text:

after

} + name: + - type: TextNode + value: p + position: { start: [0, 50], text: p } + attributes: [] + children: + - type: TextNode + value: after + position: { start: [0, 52], text: after } + - name: closes a script body after a raw {{ and a later }} around a loop + source:

after

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: '

after

' } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: '' } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: var o="{% raw %}{{{% endraw %}"; for(i=0;i } + blockEndPosition: { start: [0, 88], text:

} + position: { start: [0, 80], text:

after

} + name: + - type: TextNode + value: p + position: { start: [0, 81], text: p } + attributes: [] + children: + - type: TextNode + value: after + position: { start: [0, 83], text: after } + - name: closes a script body after raw {{ and }} split across two raw tags + source:

after

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "

after

" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: "{% raw %}{{{% endraw %} x {% raw %}}}{% endraw %}; if(a } + blockEndPosition: { start: [0, 85], text:

} + position: { start: [0, 77], text:

after

} + name: + - type: TextNode + value: p + position: { start: [0, 78], text: p } + attributes: [] + children: + - type: TextNode + value: after + position: { start: [0, 80], text: after } + - name: rejects a script body whose comment has a {{ that swallows the endcomment, as Liquid::Tokenizer does + source:

after

+ parse: [html] + error: + message: Attempting to end parsing before HtmlRawNode 'script' was closed + start: [0, 0] + text:

after

+ parse: [html] + error: + message: Attempting to end parsing before HtmlRawNode 'script' was closed + start: [0, 0] + text: + parse: [html] + error: + message: Attempting to end parsing before LiquidRawTag 'comment' was closed + start: [0, 17] + text: "{% comment %}" + - name: rejects a script with a < comparison that is never closed + source: | + + parse: [html] + error: + message: Attempting to end parsing before HtmlRawNode 'script' was closed + start: [0, 0] + text: + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: + position: { start: [0, 8], text: } + nodes: + - type: TextNode + value: + position: { start: [0, 8], text: } + - name: keeps an HTML comment in a style body as text + source: "" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlRawNode + name: style + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: css + value: "" + position: { start: [0, 7], text: "" } + nodes: + - type: TextNode + value: "" + position: { start: [0, 7], text: "" } + - name: keeps an HTML comment in an svg body as text + source: + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: } + children: + - type: HtmlRawNode + name: svg + blockStartPosition: { start: [0, 0], text: } + blockEndPosition: { start: [0, 34], text: } + position: { start: [0, 0], text: } + attributes: [] + body: + type: RawMarkup + kind: text + value: + position: { start: [0, 5], text: } + nodes: + - type: TextNode + value: + position: { start: [0, 5], text: } + - name: closes a script whose body opens an HTML comment it never closes + source:

after

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text:

after

} + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: const s = "

after

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "

after

" } + children: + - type: HtmlRawNode + name: script + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: javascript + value: /* y) {} + position: { start: [0, 8], text: "/* y) {}" } + nodes: + - type: TextNode + value: /* y) {} + position: { start: [0, 8], text: "/* y) {}" } + - type: HtmlComment + body:
+ blockStartPosition: { start: [0, 46], text: } + position: { start: [0, 46], text: } + children: + - type: TextNode + value: + position: { start: [0, 51], text: } + - type: HtmlElement + blockStartPosition: { start: [0, 61], text:

} + blockEndPosition: { start: [0, 69], text:

} + position: { start: [0, 61], text:

after

} + name: + - type: TextNode + value: p + position: { start: [0, 62], text: p } + attributes: [] + children: + - type: TextNode + value: after + position: { start: [0, 64], text: after } + - name: parses an HTML comment after a style whose body holds an unpaired /* b {}

after

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "

after

" } + children: + - type: HtmlRawNode + name: style + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + attributes: [] + body: + type: RawMarkup + kind: css + value: /* b {} + position: { start: [0, 7], text: "/* b {}" } + nodes: + - type: TextNode + value: /* b {} + position: { start: [0, 7], text: "/* b {}" } + - type: HtmlComment + body: + blockStartPosition: { start: [0, 39], text: } + position: { start: [0, 39], text: } + children: + - type: TextNode + value: + position: { start: [0, 44], text: } + - type: HtmlElement + blockStartPosition: { start: [0, 54], text:

} + blockEndPosition: { start: [0, 62], text:

} + position: { start: [0, 54], text:

after

} + name: + - type: TextNode + value: p + position: { start: [0, 55], text: p } + attributes: [] + children: + - type: TextNode + value: after + position: { start: [0, 57], text: after } + - name: closes an svg whose open tag has a glued Liquid attribute + source: + parse: [liquid] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: '' } + children: + - type: TextNode + value: + position: { start: [0, 23], text: --> } + - name: closes a nested svg at its outer when CDATA after the inner one holds a

after

+ parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text:

after

} + children: + - type: HtmlRawNode + name: svg + blockStartPosition: { start: [0, 0], text: } + blockEndPosition: { start: [0, 16], text: } + position: { start: [0, 0], text: } + attributes: [] + body: + type: RawMarkup + kind: text + value: + position: { start: [0, 5], text: } + nodes: + - type: TextNode + value: + position: { start: [0, 5], text: } + - type: HtmlComment + body: + blockStartPosition: { start: [0, 22], text: } + position: { start: [0, 22], text: } + children: + - type: TextNode + value: + position: { start: [0, 27], text: } + - type: HtmlElement + blockStartPosition: { start: [0, 37], text:

} + blockEndPosition: { start: [0, 45], text:

} + position: { start: [0, 37], text:

after

} + name: + - type: TextNode + value: p + position: { start: [0, 38], text: p } + attributes: [] + children: + - type: TextNode + value: after + position: { start: [0, 40], text: after } + - name: parses the Liquid inside an HTML comment as text and Liquid + source: + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "" } + children: + - type: HtmlComment + body: a {{ b }}

+ blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "" } + children: + - type: TextNode + value: a + position: { start: [0, 5], text: a } + - type: LiquidVariableOutput + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [0, 9], text: " b " } + position: { start: [0, 7], text: "{{ b }}" } + markup: + type: LiquidVariable + rawSource: b + position: { start: [0, 10], text: "b " } + expression: + type: VariableLookup + name: b + position: { start: [0, 10], text: b } + lookups: [] + lookupModes: [] + filters: [] + - type: TextNode + value:

+ position: { start: [0, 15], text:

} + - name: splits an HTML comment at the tag that closes an enclosing block + source: "{% for x in xs %}{% if x %}{% endfor %}" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% for x in xs %}{% if x %}{% endfor %}" } + children: + - type: LiquidTag + name: for + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 0], text: "{% for x in xs %}" } + blockEndPosition: { start: [0, 51], text: "{% endfor %}" } + markupPosition: { start: [0, 7], text: "x in xs " } + position: { start: [0, 0], text: "{% for x in xs %}{% if x %}{% endfor %}" } + markup: + type: ForMarkup + variableName: x + reversed: false + position: { start: [0, 7], text: "x in xs " } + collection: + type: VariableLookup + name: xs + position: { start: [0, 12], text: xs } + lookups: [] + lookupModes: [] + args: [] + children: + - type: LiquidBranch + name: null + markup: "" + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [0, 17], text: "" } + blockStartPosition: { start: [0, 17], text: "" } + blockEndPosition: { start: [0, 51], text: "" } + position: { start: [0, 17], text: "{% if x %}" } + children: + - type: LiquidTag + name: if + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 17], text: "{% if x %}" } + blockEndPosition: { start: [0, 34], text: "{% endif %}" } + markupPosition: { start: [0, 23], text: "x " } + position: { start: [0, 17], text: "{% if x %} } + position: { start: [0, 45], text: " b -->" } + children: + - type: TextNode + value: b + position: { start: [0, 46], text: b } + - name: ends an HTML comment with a block opened inside it that closes after it + source: a {% endif %}c + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: " a {% endif %}c" } + children: + - type: HtmlComment + body: "{% if x %} --> a {% endif %}" + blockStartPosition: { start: [0, 0], text: a {% endif %}" } + children: + - type: LiquidTag + name: if + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 5], text: "{% if x %}" } + blockEndPosition: { start: [0, 22], text: "{% endif %}" } + markupPosition: { start: [0, 11], text: "x " } + position: { start: [0, 5], text: "{% if x %} --> a {% endif %}" } + markup: + type: VariableLookup + name: x + position: { start: [0, 11], text: x } + lookups: [] + lookupModes: [] + children: + - type: LiquidBranch + name: null + markup: "" + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [0, 15], text: "" } + blockStartPosition: { start: [0, 15], text: "" } + blockEndPosition: { start: [0, 22], text: "" } + position: { start: [0, 15], text: " --> a " } + children: + - type: TextNode + value: --> a + position: { start: [0, 16], text: --> a } + - type: HtmlElement + blockStartPosition: { start: [0, 33], text: } + blockEndPosition: { start: [0, 37], text: } + position: { start: [0, 33], text: c } + name: + - type: TextNode + value: b + position: { start: [0, 34], text: b } + attributes: [] + children: + - type: TextNode + value: c + position: { start: [0, 36], text: c } + - name: ends an HTML comment that never closes at the end of the source, as a browser does + source: a after + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "after" } + children: + - type: HtmlComment + body: "{% if true %}--> } + position: { start: [0, 0], text: "" } + children: + - type: LiquidTag + name: if + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 4], text: "{% if true %}" } + blockEndPosition: { start: [0, 24], text: "{% endif %}" } + markupPosition: { start: [0, 10], text: "true " } + position: { start: [0, 4], text: "{% if true %}-->{% endif %}" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% if a %}{% endif %}" } + children: + - type: LiquidTag + name: if + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 0], text: "{% if a %}" } + blockEndPosition: { start: [0, 33], text: "{% endif %}" } + markupPosition: { start: [0, 6], text: "a " } + position: { start: [0, 0], text: "{% if a %}{% endif %}" } + markup: + type: VariableLookup + name: a + position: { start: [0, 6], text: a } + lookups: [] + lookupModes: [] + children: + - type: LiquidBranch + name: null + markup: "" + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [0, 10], text: "" } + blockStartPosition: { start: [0, 10], text: "" } + blockEndPosition: { start: [0, 17], text: "" } + position: { start: [0, 10], text: "" } + children: + - type: HtmlComment + body: y + blockStartPosition: { start: [0, 27], text: "" } + blockEndPosition: { start: [0, 30], text: --> } + position: { start: [0, 27], text: " y -->" } + children: + - type: TextNode + value: y + position: { start: [0, 28], text: y } + - name: rejects an orphan end tag inside an HTML comment, as Liquid does + source: + parse: [html] + error: + message: Attempting to close LiquidTag 'if' before it was opened without a matching 'if' + start: [0, 5] + text: "{% endif %}" + - name: rejects an unclosed block inside an HTML comment, as Liquid does + source: + parse: [html] + error: + message: Attempting to end parsing before LiquidTag 'if' was closed + start: [0, 5] + text: "{% if a %}" + - name: parses a raw tag inside an HTML comment as a raw tag + source: {% endraw %} --> + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% endraw %} -->" } + children: + - type: HtmlComment + body: "{% raw %}{{ a }} -->{% endraw %}" + blockStartPosition: { start: [0, 0], text: } + position: { start: [0, 0], text: "{% endraw %} -->" } + children: + - type: LiquidRawTag + name: raw + markup: "" + markupPosition: { start: [0, 12], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 5], text: "{% raw %}" } + blockEndPosition: { start: [0, 25], text: "{% endraw %}" } + position: { start: [0, 5], text: "{% raw %}{{ a }} -->{% endraw %}" } + body: + type: RawMarkup + kind: text + value: "{{ a }} -->" + position: { start: [0, 14], text: "{{ a }} -->" } + nodes: + - type: TextNode + value: "{{ a }} -->" + position: { start: [0, 14], text: "{{ a }} -->" } diff --git a/specs/parser/liquid-lines.yml b/specs/parser/liquid-lines.yml index 6abbf566a..830b7de80 100644 --- a/specs/parser/liquid-lines.yml +++ b/specs/parser/liquid-lines.yml @@ -1931,7 +1931,7 @@ cases: comment inner endcomment - - name: should carve out a nested raw so a literal endcomment inside it does not close early + - name: should leave a comment unclosed when a raw inside it cannot close in {% liquid %} source: |- {% liquid comment @@ -1961,6 +1961,14 @@ cases: children: - type: LiquidTag name: liquid + markup: |- + comment + before + raw + endcomment + endraw + after + endcomment whitespaceStart: "" whitespaceEnd: "" blockStartPosition: @@ -1998,6 +2006,69 @@ cases: after endcomment %} + reason: Unclosed raw tag 'comment' in {% liquid %} block + - name: should close a nested comment at the endcomment of its outer level + source: |- + {% liquid + comment + comment + endcomment + echo 'X' + endcomment + echo 'Y' + %} + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + {% liquid + comment + comment + endcomment + echo 'X' + endcomment + echo 'Y' + %} + children: + - type: LiquidTag + name: liquid + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: + start: [0, 0] + text: |- + {% liquid + comment + comment + endcomment + echo 'X' + endcomment + echo 'Y' + %} + markupPosition: + start: [0, 9] + text: | + + comment + comment + endcomment + echo 'X' + endcomment + echo 'Y' + position: + start: [0, 0] + text: |- + {% liquid + comment + comment + endcomment + echo 'X' + endcomment + echo 'Y' + %} markup: - type: LiquidRawTag name: comment @@ -2008,48 +2079,355 @@ cases: delimiterWhitespaceStart: "" delimiterWhitespaceEnd: "" blockStartPosition: { start: [1, 0], text: comment } - blockEndPosition: { start: [7, 0], text: endcomment } + blockEndPosition: { start: [5, 0], text: endcomment } position: start: [1, 0] text: |- comment - before - raw + comment endcomment - endraw - after + echo 'X' endcomment body: type: RawMarkup kind: text value: | - before - raw + comment endcomment - endraw - after + echo 'X' position: start: [1, 7] text: | - before - raw + comment endcomment - endraw - after + echo 'X' nodes: - type: TextNode value: |- - before - raw + comment endcomment - endraw - after + echo 'X' position: start: [2, 0] text: |- - before - raw + comment endcomment - endraw - after + echo 'X' + - type: LiquidTag + name: echo + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: { start: [6, 0], text: echo 'Y' } + markupPosition: { start: [6, 5], text: "'Y'" } + position: { start: [6, 0], text: echo 'Y' } + markup: + type: LiquidVariable + rawSource: "'Y'" + position: { start: [6, 5], text: "'Y'" } + expression: + type: String + single: true + value: Y + position: { start: [6, 5], text: "'Y'" } + filters: [] + - name: should not count a comment line indented with non-ASCII whitespace + source: |- + {% liquid + comment +  comment + endcomment + echo 'Y' + %} + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + {% liquid + comment +  comment + endcomment + echo 'Y' + %} + children: + - type: LiquidTag + name: liquid + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: + start: [0, 0] + text: |- + {% liquid + comment +  comment + endcomment + echo 'Y' + %} + markupPosition: + start: [0, 9] + text: | + + comment +  comment + endcomment + echo 'Y' + position: + start: [0, 0] + text: |- + {% liquid + comment +  comment + endcomment + echo 'Y' + %} + markup: + - type: LiquidRawTag + name: comment + markup: "" + markupPosition: { start: [1, 7], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [1, 0], text: comment } + blockEndPosition: { start: [3, 0], text: endcomment } + position: + start: [1, 0] + text: |- + comment +  comment + endcomment + body: + type: RawMarkup + kind: text + value: | +  comment + position: + start: [1, 7] + text: | + +  comment + nodes: + - type: TextNode + value: comment + position: { start: [2, 1], text: comment } + - type: LiquidTag + name: echo + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: { start: [4, 0], text: echo 'Y' } + markupPosition: { start: [4, 5], text: "'Y'" } + position: { start: [4, 0], text: echo 'Y' } + markup: + type: LiquidVariable + rawSource: "'Y'" + position: { start: [4, 5], text: "'Y'" } + expression: + type: String + single: true + value: Y + position: { start: [4, 5], text: "'Y'" } + filters: [] + - name: should not count a raw line indented with non-ASCII whitespace + source: |- + {% liquid + comment +  raw + endcomment + echo 'Y' + %} + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + {% liquid + comment +  raw + endcomment + echo 'Y' + %} + children: + - type: LiquidTag + name: liquid + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: + start: [0, 0] + text: |- + {% liquid + comment +  raw + endcomment + echo 'Y' + %} + markupPosition: + start: [0, 9] + text: | + + comment +  raw + endcomment + echo 'Y' + position: + start: [0, 0] + text: |- + {% liquid + comment +  raw + endcomment + echo 'Y' + %} + markup: + - type: LiquidRawTag + name: comment + markup: "" + markupPosition: { start: [1, 7], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [1, 0], text: comment } + blockEndPosition: { start: [3, 0], text: endcomment } + position: + start: [1, 0] + text: |- + comment +  raw + endcomment + body: + type: RawMarkup + kind: text + value: | +  raw + position: + start: [1, 7] + text: | + +  raw + nodes: + - type: TextNode + value: raw + position: { start: [2, 1], text: raw } + - type: LiquidTag + name: echo + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: { start: [4, 0], text: echo 'Y' } + markupPosition: { start: [4, 5], text: "'Y'" } + position: { start: [4, 0], text: echo 'Y' } + markup: + type: LiquidVariable + rawSource: "'Y'" + position: { start: [4, 5], text: "'Y'" } + expression: + type: String + single: true + value: Y + position: { start: [4, 5], text: "'Y'" } + filters: [] + - name: should not count an endcomment line indented with non-ASCII whitespace + source: |- + {% liquid + comment +  endcomment + endcomment + echo 'Y' + %} + parse: [html] + ast: + type: Document + name: "#document" + position: + start: [0, 0] + text: |- + {% liquid + comment +  endcomment + endcomment + echo 'Y' + %} + children: + - type: LiquidTag + name: liquid + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: + start: [0, 0] + text: |- + {% liquid + comment +  endcomment + endcomment + echo 'Y' + %} + markupPosition: + start: [0, 9] + text: | + + comment +  endcomment + endcomment + echo 'Y' + position: + start: [0, 0] + text: |- + {% liquid + comment +  endcomment + endcomment + echo 'Y' + %} + markup: + - type: LiquidRawTag + name: comment + markup: "" + markupPosition: { start: [1, 7], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [1, 0], text: comment } + blockEndPosition: { start: [3, 0], text: endcomment } + position: + start: [1, 0] + text: |- + comment +  endcomment + endcomment + body: + type: RawMarkup + kind: text + value: | +  endcomment + position: + start: [1, 7] + text: | + +  endcomment + nodes: + - type: TextNode + value: endcomment + position: { start: [2, 1], text: endcomment } + - type: LiquidTag + name: echo + whitespaceStart: "" + whitespaceEnd: "" + blockStartPosition: { start: [4, 0], text: echo 'Y' } + markupPosition: { start: [4, 5], text: "'Y'" } + position: { start: [4, 0], text: echo 'Y' } + markup: + type: LiquidVariable + rawSource: "'Y'" + position: { start: [4, 5], text: "'Y'" } + expression: + type: String + single: true + value: Y + position: { start: [4, 5], text: "'Y'" } + filters: [] diff --git a/specs/parser/liquid-raw.yml b/specs/parser/liquid-raw.yml index 31febb4d9..33acdff5c 100644 --- a/specs/parser/liquid-raw.yml +++ b/specs/parser/liquid-raw.yml @@ -667,7 +667,7 @@ cases: - type: TextNode value: position: { start: [0, 18], text: } - - name: should depth-balance nested same-name raw tags and close at the outer tag (issue 156) (1) + - name: should depth-balance nested same-name raw tags and close at the outer tag (issue 156) source: abc parse: [html] ast: @@ -690,29 +690,13 @@ cases: - type: TextNode value: abc position: { start: [0, 5], text: abc } - - name: should depth-balance nested same-name raw tags and close at the outer tag (issue 156) (2) + - name: closes a script at its first close tag, as a browser does, so a nested script's close is the script's source: c parse: [html] - ast: - type: Document - name: "#document" - position: { start: [0, 0], text: c } - children: - - type: HtmlRawNode - name: script - blockStartPosition: { start: [0, 0], text: } - position: { start: [0, 0], text: c } - attributes: [] - body: - type: RawMarkup - kind: javascript - value: ac - position: { start: [0, 8], text: ac } - nodes: - - type: TextNode - value: ac - position: { start: [0, 8], text: ac } + error: + message: Attempting to close HtmlElement 'script' before it was opened + start: [0, 28] + text: {{ x }}" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% comment %}a{% comment %}b{% endcomment %}c{% endcomment %}{{ x }}" } + children: + - type: LiquidRawTag + name: comment + markup: "" + markupPosition: { start: [0, 11], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 0], text: "{% comment %}" } + blockEndPosition: { start: [0, 45], text: "{% endcomment %}" } + position: { start: [0, 0], text: "{% comment %}a{% comment %}b{% endcomment %}c{% endcomment %}" } + body: + type: RawMarkup + kind: text + value: a{% comment %}b{% endcomment %}c + position: { start: [0, 13], text: "a{% comment %}b{% endcomment %}c" } + nodes: + - type: TextNode + value: a{% comment %}b{% endcomment %}c + position: { start: [0, 13], text: "a{% comment %}b{% endcomment %}c" } + - type: HtmlElement + blockStartPosition: { start: [0, 61], text: } + blockEndPosition: { start: [0, 75], text: } + position: { start: [0, 61], text: "{{ x }}" } + name: + - type: TextNode + value: title + position: { start: [0, 62], text: title } + attributes: [] + children: + - type: LiquidVariableOutput + whitespaceStart: "" + whitespaceEnd: "" + markupPosition: { start: [0, 70], text: " x " } + position: { start: [0, 68], text: "{{ x }}" } + markup: + type: LiquidVariable + rawSource: x + position: { start: [0, 71], text: "x " } + expression: + type: VariableLookup + name: x + position: { start: [0, 71], text: x } + lookups: [] + lookupModes: [] + filters: [] + - name: parses the markup after a comment whose body opens a tag + source: "{% comment %} ax" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% comment %} ax" } + children: + - type: LiquidRawTag + name: comment + markup: "" + markupPosition: { start: [0, 11], text: "" } + whitespaceStart: "" + whitespaceEnd: "" + delimiterWhitespaceStart: "" + delimiterWhitespaceEnd: "" + blockStartPosition: { start: [0, 0], text: "{% comment %}" } + blockEndPosition: { start: [0, 18], text: "{% endcomment %}" } + position: { start: [0, 0], text: "{% comment %} a } + blockEndPosition: { start: [0, 40], text: } + position: { start: [0, 34], text:

x
} + name: + - type: TextNode + value: div + position: { start: [0, 35], text: div } + attributes: [] + children: + - type: TextNode + value: x + position: { start: [0, 39], text: x } + - name: parses the markup after a comment whose body opens a quoted attribute + source: '{% comment %} x" + parse: [html] + ast: + type: Document + name: "#document" + position: { start: [0, 0], text: "{% comment %}