Skip to content

String substitution in scmOverrides replace - #720

Merged
jkloetzke merged 3 commits into
BobBuildTool:masterfrom
MarcKe:mk_replace_string_substitution
Oct 8, 2026
Merged

jkloetzke merged 3 commits into
BobBuildTool:masterfrom
MarcKe:mk_replace_string_substitution

Conversation

@MarcKe

@MarcKe MarcKe commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

string substitution was not available for replace in scmOverrides.
since it can break some old overrides, it is hidded behind a new policy flag.

@codecov

codecov Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.89%. Comparing base (7baba22) to head (b046116).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #720   +/-   ##
=======================================
  Coverage   89.89%   89.89%           
=======================================
  Files          50       50           
  Lines       16897    16901    +4     
=======================================
+ Hits        15189    15193    +4     
  Misses       1708     1708           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jkloetzke jkloetzke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you give a use case where this is needed? So far I've never came across the need to replace things in overrides dynamically...

Comment thread doc/manual/configuration.rst Outdated
Comment on lines +3198 to +3202
substitution is applied to the regular expression before it is compiled. A
literal ``$`` in a ``pattern`` (e.g. the end-of-string anchor) must
therefore be escaped as ``\$``. Be aware that YAML double quoted strings
interpret backslashes themselves, so write ``'^.*\$'`` (single quotes) or
``"^.*\\$"``.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly why I don't think this is something that should be enabled by default. Writing a regex pattern or replacement (with back references) is already hard enough in Yaml. Additionally substituting the pattern and replacement by Bob makes it very hard to use. How about making this a per-override setting (e.g. replaceSubst: True)?

@MarcKe

MarcKe commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Could you give a use case where this is needed? So far I've never came across the need to replace things in overrides dynamically...

Ok, I can understand why it might be weird for pattern, but replacement should definitely have this feature.

match:
    url: "https://awesometools.org/tools/*"
replace:
    url:
        pattern: "https://awesometools.org/"
        replacement: "${INTERNAL_MIRROR}/awesometools/"

for pattern, maybe the following is a bit of an artificial usecase:

match:
    url: "${GNU_MIRROR}/{tool_a, tool_b}/*"
replace:
    url:
        pattern: "${GNU_MIRROR}"
        replacement: "${INTERNAL_MIRROR}/some_awesome_tools/"
match:
    url: "${GNU_MIRROR}/*/*"
replace:
    url:
        pattern: "${GNU_MIRROR}"
        replacement: "${INTERNAL_MIRROR}/other_place_for_awesome_tools/"

Not sure if this is 100% valid.

I would agree to making string substitution for pattern configurable, but replacement should always have this.

MarcKe added 3 commits October 8, 2026 11:31
pattern and replacement will now be subject to string substitution,
if the per override setting `substituteReplace` is set to True.
default is False to keep the old behaviour.

dropped the eager early __init in scm.py. __applyEnv now substitutes
`pattern` and `replacement` and then compiles the regexes.
@MarcKe
MarcKe force-pushed the mk_replace_string_substitution branch from d6ba6b2 to b046116 Compare October 8, 2026 09:38
@MarcKe

MarcKe commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

Could you give a use case where this is needed? So far I've never came across the need to replace things in overrides dynamically...

Ok, I can understand why it might be weird for pattern, but replacement should definitely have this feature.

match:
    url: "https://awesometools.org/tools/*"
replace:
    url:
        pattern: "https://awesometools.org/"
        replacement: "${INTERNAL_MIRROR}/awesometools/"

for pattern, maybe the following is a bit of an artificial usecase:

match:
    url: "${GNU_MIRROR}/{tool_a, tool_b}/*"
replace:
    url:
        pattern: "${GNU_MIRROR}"
        replacement: "${INTERNAL_MIRROR}/some_awesome_tools/"
match:
    url: "${GNU_MIRROR}/*/*"
replace:
    url:
        pattern: "${GNU_MIRROR}"
        replacement: "${INTERNAL_MIRROR}/other_place_for_awesome_tools/"

Not sure if this is 100% valid.

I would agree to making string substitution for pattern configurable, but replacement should always have this.

Ok, I have used the per override setting for pattern and replacement in the end.

@jkloetzke

Copy link
Copy Markdown
Member

Thanks for giving the rationale. LGTM.

@jkloetzke
jkloetzke merged commit 9624e4d into BobBuildTool:master Oct 8, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants