The grammar accepts escaped string delimiters inside a template directive, which Terraform rejects

[livingstaccato]

Summary

Inside a template directive, this grammar accepts a string literal written with
escaped delimiters -- "%{ if x == \"y\" }t%{ endif }" -- through the
TEMPLATE_STRING terminal. OpenTofu rejects that source outright.

Terraform writes such a literal with plain delimiters: "%{ if x == "y" }t%{ endif }". The braces let its scanner track the nesting, so no escaping is
needed or accepted.

Reproduction

import hcl2

hcl2.loads('a = "%{ if x == \\"y\\" }t%{ endif }"\n')
# {'a': '"%{ if x == \\"y\\" }t%{ endif }"'}     accepted

The same file in OpenTofu v1.12.5:

Error: Invalid character

  on main.tf line 3, in locals:
   3:   a = "%{ if local.x == \"y\" }t%{ endif }"

The plain spelling is accepted by both, and evaluates to "t".

Why it is worth a decision rather than a patch

Accepting more than the reference implementation is a milder fault than
rejecting valid input. But it means a document can parse here and fail in
Terraform, which is the direction that costs a user the most time -- the file
looks fine until it reaches the tool that matters.

The fixture covers the escaped form, in a case named for the issue it came
from:

issue_247 = "kms%{ if var.id != \"primary\" }-${var.id}%{ endif }"

That naming is misleading, and it is worth correcting before anyone weighs this.
#247 reported the plain spelling:

locals {
  service = "kms%{ if var.id != "primary" }-${var.id}%{ endif }"
}

The escaped version appears only in the fixture. Commit 5473741 ("Add
template directives support (%{if}, %{for}) in quoted strings", #276) added the
TEMPLATE_STRING terminal and that fixture line together, and the terminal's
pattern matches escaped delimiters specifically. So the shape being supported
is not the shape the reporter had; it reads like the config was transcribed
through a Python string literal and the escaping came along with it.

Removing the terminal is a breaking change to public API, not a fixture update.

• On the read side it does what you would want: the plain spelling from Parsing error when comparing strings within a string template directive #247
  still parses, the escaped one stops, and the suite needs four fixture/expected
  files updated.
• On the write side it breaks dumps, which is exported and documented. 8.1.3
  serializes its own template_directives.tf fixture to
  {"issue_247": "\"kms%{ if var.id != \\\"primary\\\" }-${var.id}%{ endif }\""},
  and feeding that JSON back raises lark.exceptions.UnexpectedToken. Stored
  8.1.x JSON therefore stops converting, and jsontohcl2 --skip cannot skip it:
  JSON_SKIPPABLE is only (JSONDecodeError, UnicodeDecodeError).

Repairing the old spelling on read -- unescaping the delimiters rather than
refusing them -- is possible in principle and would turn the break into a
migration. It needs a scanner that tracks nested string literals and backslash
parity, though: a naive one corrupts ${upper("a\"b")}, where the escape is
legitimate. I tried the naive version and threw it away.

The grammar change and the fixture updates are on
livingstaccato/python-hcl2:fix/escaped-delimiters-in-directives if the diff is
useful. I have not opened a pull request, because what this costs is a
compatibility decision rather than a technical one, and that call is yours --
including deciding the escaped spelling is worth keeping.

Filed separately from #341, which only stops the value form from mangling this
spelling into a bare reference; that fix is correct whichever way this goes.

---

This issue, and the investigation behind it, were produced by an AI assistant (Claude) working on behalf of the author. Please review with that provenance in mind.