Summary
HEREDOC_TEMPLATE (hcl2/hcl2.lark:92 at v8.1.3) ends
\s*(?P=heredoc)\r?\n, requiring the newline to follow the delimiter word
immediately, so a marker line with a trailing space or tab is not recognised
and the parse fails. HEREDOC_TEMPLATE_TRIM at :93 ends the same way. Terraform's scanner ends the heredoc there.
Reproduction
hcl2.loads("a = <<EOF\nbody\nEOF \nb = 1\n")
# lark.exceptions.UnexpectedToken: Unexpected token Token('STRING_CHARS', ...)The same file in OpenTofu evaluates a to "body\n" and b to 1 -- the
EOF line closes the heredoc.
Trailing whitespace is invisible and survives copy-paste and editors that do
not trim it, so a file a user has been running through Terraform for months
can fail here with an error that points at the wrong line.
Suggested shape of the fix
Allow horizontal whitespace after the delimiter in both heredoc terminals:
\s*(?P=heredoc)[ \t]*\r?\n.
This issue, and the investigation behind it, were produced by an AI assistant (Claude) working on behalf of the author. Please review with that provenance in mind.
Summary
HEREDOC_TEMPLATE(hcl2/hcl2.lark:92atv8.1.3) ends\s*(?P=heredoc)\r?\n, requiring the newline to follow the delimiter wordimmediately, so a marker line with a trailing space or tab is not recognised
and the parse fails.
HEREDOC_TEMPLATE_TRIMat:93ends the same way. Terraform's scanner ends the heredoc there.Reproduction
The same file in OpenTofu evaluates
ato"body\n"andbto1-- theEOFline closes the heredoc.Trailing whitespace is invisible and survives copy-paste and editors that do
not trim it, so a file a user has been running through Terraform for months
can fail here with an error that points at the wrong line.
Suggested shape of the fix
Allow horizontal whitespace after the delimiter in both heredoc terminals:
\s*(?P=heredoc)[ \t]*\r?\n.This issue, and the investigation behind it, were produced by an AI assistant (Claude) working on behalf of the author. Please review with that provenance in mind.