Skip to content

Commit c580e72

Browse files
committed
Fix tokenizer to skip closing braces inside quoted strings
The byte-scanning tokenizer does not account for quoted strings when scanning for `}}` to terminate a variable token. A `}` inside a quoted filter argument prematurely ends the token, causing a SyntaxError. For example, the following valid template fails to parse: {{ message | replace: "{name}", customer_name }} This patch makes `next_variable_token` skip over single- and double-quoted strings so that their contents do not interfere with `}}` detection.
1 parent dd37353 commit c580e72

2 files changed

Lines changed: 18 additions & 1 deletion

File tree

lib/liquid/tokenizer.rb

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ class Tokenizer
1313
OPEN_CURLEY = "{".ord
1414
CLOSE_CURLEY = "}".ord
1515
PERCENTAGE = "%".ord
16+
SINGLE_QUOTE = "'".ord
17+
DOUBLE_QUOTE = '"'.ord
1618

1719
def initialize(
1820
source:,
@@ -117,10 +119,19 @@ def next_variable_token
117119
byte_a = byte_b = @ss.scan_byte
118120

119121
while byte_b
120-
byte_a = @ss.scan_byte while byte_a && byte_a != CLOSE_CURLEY && byte_a != OPEN_CURLEY
122+
byte_a = @ss.scan_byte while byte_a &&
123+
byte_a != CLOSE_CURLEY && byte_a != OPEN_CURLEY &&
124+
byte_a != SINGLE_QUOTE && byte_a != DOUBLE_QUOTE
121125

122126
break unless byte_a
123127

128+
if byte_a == SINGLE_QUOTE || byte_a == DOUBLE_QUOTE
129+
@ss.skip_until(byte_a == SINGLE_QUOTE ? /'/ : /"/)
130+
131+
byte_a = @ss.scan_byte
132+
next
133+
end
134+
124135
if @ss.eos?
125136
return byte_a == CLOSE_CURLEY ? @source.byteslice(start, @ss.pos - start) : "{{"
126137
end

test/unit/tokenizer_unit_test.rb

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,12 @@ def test_incomplete_curly_braces
4141
assert_equal(["{{}}", "}"], tokenize('{{}}}'))
4242
end
4343

44+
def test_closing_brace_in_quoted_string
45+
assert_equal(['{{ msg | replace: "{name}", name }}'], tokenize('{{ msg | replace: "{name}", name }}'))
46+
assert_equal(["{{ msg | replace: '{name}', name }}"], tokenize("{{ msg | replace: '{name}', name }}"))
47+
assert_equal(['{{ x | prepend: " {{ " | append: " }} " }}'], tokenize('{{ x | prepend: " {{ " | append: " }} " }}'))
48+
end
49+
4450
def test_unmatching_start_and_end
4551
assert_equal(["{{%}"], tokenize('{{%}'))
4652
assert_equal(["{{%%%}}"], tokenize('{{%%%}}'))

0 commit comments

Comments
 (0)