rego: Fragment specialization#2789
Open
micromaomao wants to merge 9 commits into
Open
Conversation
In implementing the fragment parameters feature, we need a way to store, for
each issuer and feed tuple, which parameters should be used for fragments
matching it. Consider this list of fragments, which a parent fragment might
define:
fragments := [
{
"feed": "mcr.microsoft.com/maa/enclavehost",
"issuer": "did:x509:0:sha256:...",
"includes": [
"containers",
"fragments"
],
"minimum_svn": "1",
"parameters": {
"region": "australiacentral2",
"cloud": "Public"
}
},
// ...
]
When we load the fragment containing this, we need to iterate through its
data.<namespace>.fragments array, and for each entry, append the parameters
object to an array that is keyed by the issuer and feed, since we can have
multiple such fragment import entries for the same issuer and feed, but with
different parameters.
This basically means that we need the equivalent of the following code in Rego,
which I've failed to come up with a way to write:
for f in fragment_fragments {
issuer = data.metadata.issuers[f.issuer] or {}
feed = issuer.feeds[f.feed] or {}
feed.parameters = feed.parameters or []
feed.parameters = array_append(feed.parameters, f.parameters)
issuer.feeds[f.feed] = feed
data.metadata.issuers[f.issuer] = issuer
}
While we can dynamically lookup or store based on a key that is from a variable
access, and we can send multiple metadata updates, those updates cannot express
the semantic of "merging" objects or arrays.
To make this simpler, we introduce a new metadata data type - "set", which
supports storing an unordered list of arbitrary objects (which themselves might
be either a string, or an object containing key-value pairs), that can be
inserted to via metadata operations one at a time. This means that we can
simply append to the metadata update operations array once per each fragment
import statement, and the outcome would be the union of all the parameter
objects. This is also very easy to query in Rego, given an issuer and feed:
parameters := [
fp.parameters |
fp = data.metadata.fragment_parameters[_]
fp.issuer == input.issuer
fp.feed == input.feed
]
It is likely that the existing code that extracts included containers/fragments
from a fragment can be simplified by using this feature, but that is outside of
the scope of this PR.
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…and improve comments Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…ching strategies This is to make it easier to parameterize environment rules. Currently, name and value for an environment rule are actually combined into one "pattern" field, and there is only one strategy for the combined pattern. This presents a problem when a fragment wants to delegate the decision of e.g. whether to match the value (but only the value, not the key) with a regex or with a fixed string. We split "pattern" and "strategy" out into "name", "name_strategy", "value" and "value_strategy" in order to allow more flexibility when fragment exposes env-var parameters. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
This commit implements the support for the "parameters" feature for fragments.
This is achieved by having the framework extract the parameters object from
fragment import statements, and storing them in a set, tagged with the
applicable (issuer, feed) pair. On future fragment loads, if the fragment's
issuer and feed pair matches with any previous entry from this set, the
parameters will be passed to the fragment via an automatically injected object
added to the end of the fragment's Rego source (__fragment_parameters).
(Note that in reality, we need to combine this set with the import statements
defined in the main policy. c.f. candidate_fragments. This is done in
fragment_parameters_for)
In order for fragments to use this parameter object, it is expected that all
fragments will now have an additional "stub" inserted at runtime to define the
parameter() function, which will, under the hood, reference a "hidden" variable
__fragment_parameters, also inserted dynamically at runtime, containing the
actual parameter values.
If multiple fragment parameters are specified, all combinations of values are
considered "allowed", and therefore the fragment injection is repeated, passing
in different parameter combinations, one for each combination. This means that
if a fragment defines, for example:
containers := [
{
...
"env_rules": [
{
"name": "SOME_KEY",
"name_strategy": "string",
"value": parameter("my_param"),
"value_strategy": "string"
}
]
}
]
and we have two fragment import statement for this fragment, e.g.:
fragments := [
{
"feed": ...,
"issuer": ...,
"minimum_svn": ...,
"parameters": {
"my_param": "value1"
}
},
{
"feed": ...,
"issuer": ...,
"minimum_svn": ...,
"parameters": {
"my_param": "value2"
}
}
]
Then the container can be started with either SOME_KEY=value1 or
SOME_KEY=value2.
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
---
Changes:
- Fix missing [_]
- fix wrong usage of defer
- Inject fragment parameter function definitions at runtime:
We must inject this at runtime so as to avoid having to support this exact
implementation (eg __fragment_parameters[name]) of fetching the parameters
forever (as it will essentially be hard-coded in the generated policy).
- Move the actual implementation of parameter() into the framework so that
fragments doesn't have to say 'import future.keywords.every', 'import
future.keywords.in'.
- Use indirection with default when accessing fragment.parameters to not break
older fragments.
If we don't do this, loading a fragment without the parameters definition
would fail due to "unsafe" rego variable references.
- Fix broken tests caused by the new extract_parameter framework function
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
…ing env list If every element in the input env list matches some env rule in some container, but there is no container that matches the entire env list, we currently deny correctly but returns no error message. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
This will test for scenarios like different fragments using the same parameter name, multiple parameter combinations, correct parameter passing for nested fragments, and make sure container creation is denied if the parameter and the given environment / command doesn't match. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
Name still provisional, might change later. Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>
There was a problem hiding this comment.
Pull request overview
This PR adds “fragment specialization” (fragment parameters) to the Rego security policy framework by allowing fragment import statements to provide a parameters object, injecting those parameters (and a parameter() helper) into fragment Rego at load time, and extending fragment metadata handling to support nested fragments and parameter propagation.
Changes:
- Injects fragment parameter definitions into fragment Rego at load time and loads fragments once per allowed parameter set.
- Extends
framework.regoto (a) support env-rules expressed as eitherpattern/strategyorname/valuewith separate strategies, and (b) propagate fragment parameters via metadata. - Adds “set” support to the rego policy interpreter metadata operations and introduces extensive Linux tests + embedded fragment policy fixtures.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/securitypolicy/version_framework | Bumps framework version to 0.5.0. |
| pkg/securitypolicy/securitypolicyenforcer_rego.go | Injects parameter support into fragment modules and iterates fragment loading across parameter possibilities. |
| pkg/securitypolicy/securitypolicy.go | Extends EnvRuleConfig with name/value-based env-rule fields. |
| pkg/securitypolicy/securitypolicy_marshal.go | Marshals new env-rule form and adds fragment parameters to fragment import emission. |
| pkg/securitypolicy/securitypolicy_internal.go | Extends internal fragment representation to carry parameters. |
| pkg/securitypolicy/framework.rego | Adds env-rule dual-form matching and fragment-parameter metadata plumbing + helper extraction rule. |
| pkg/securitypolicy/fragment_definition.rego | Defines injected parameter() helper and fragment parameter metadata accessor. |
| pkg/securitypolicy/regopolicy_linux_test.go | Updates existing tests and adds comprehensive fragment-parameter test coverage with embedded Rego fixtures. |
| pkg/securitypolicy/rego_utils_test.go | Updates env var generators/helpers to better model NAME=VALUE strings and env-rule variations. |
| pkg/securitypolicy/fragment_test_policies/_container_common.rego.inc | Shared fixture include for fragment tests. |
| pkg/securitypolicy/fragment_test_policies/simple_env_rule_param.rego | Fixture fragment consuming an env-rule parameter. |
| pkg/securitypolicy/fragment_test_policies/env_rule_param.rego | Fixture fragment consuming multiple parameter shapes (object + string). |
| pkg/securitypolicy/fragment_test_policies/env_rule_param_another_fragment.rego | Fixture fragment for multi-fragment/parameter name collision tests. |
| pkg/securitypolicy/fragment_test_policies/nested_importer.rego | Fixture fragment that imports nested fragments with parameterized parameters. |
| pkg/securitypolicy/fragment_test_policies/nested_importer_2.rego | Second nested-importer fixture to expand parameter combinations. |
| pkg/securitypolicy/fragment_test_policies/nested_fragment.rego | Nested fragment fixture consuming parameters with defaults. |
| pkg/securitypolicy/fragment_test_policies/param_on_command.rego | Fixture fragment parameterizing command arrays. |
| internal/regopolicyinterpreter/regopolicyinterpreter.go | Adds metadata “set” type support and an Array accessor on query results. |
| internal/regopolicyinterpreter/test.rego | Adds test helpers for set add/remove/contains/get. |
| internal/regopolicyinterpreter/regopolicyinterpreter_test.go | Adds property tests covering set metadata semantics and updates metadata copy assertions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+113
to
+117
| UseNameValue bool | ||
| Name string | ||
| NameStrategy EnvVarRule | ||
| Value string | ||
| ValueStrategy EnvVarRule |
Comment on lines
+8449
to
+8454
| for i_fragment, f := range fragmentsToLoad { | ||
| err = policy.LoadFragment(gc.ctx, topFragmentIssuer, fmt.Sprintf(topFragmentFeedFmt, i_fragment), paramTestTemplateFragmentCode(f.fragmentCode)) | ||
| } | ||
| if err != nil { | ||
| t.Fatalf("failed to load fragment: %v", err) | ||
| } |
Comment on lines
+52
to
+56
| maxGeneratedEnvironmentVariables = 16 | ||
| maxGeneratedEnvironmentVariableNameLength = 31 | ||
| maxGeneratedEnvironmentVariableValueLength = 32 | ||
| maxGeneratedEnvironmentVariableRules = 8 | ||
| maxGeneratedFragmentNamespaceLength = 32 |
Comment on lines
+580
to
+583
| paramsJson, err := json.Marshal(f.parameters) | ||
| if err != nil { | ||
| panic(fmt.Errorf("failed to marshal fragment parameters object to JSON: %w", err)) | ||
| } |
Comment on lines
+237
to
+238
| # A env rule can be of two form: | ||
| # { |
Member
Author
|
@KenGordon I was discussing with Takuro on whether this scheme would allow a main policy writer to say "allow any values from set A for parameter A, and allow any values from set B for parameter B". Currently this requires specifying all combinations of allowed parameters, so they would have to have Do you think the current scheme is flexible enough as it is? In theory they can allow any combination of parameter values they specify, but if their rule is very lax and they have more parameters, the number of combinations they need to allow grows exponentially. If this case would be rare then we can get away with not worrying about it, but wanted to check with you. |
Member
Author
|
@takuro-sato Ken was happy with not doing anything special for allowing set of values for now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements the "fragment specialization" aka. fragment with parameters feature, along with a new way to express env-rules using separate "name" / "value" fields.
We add a new parameters object in the fragment import statements. On fragment loads, the parameters will be passed to the fragment via an automatically injected object added to the end of the fragment's Rego source, and can be accessed using the
parameter()function, which is also runtime injected.The fragment itself also needs to define an additional
parameters_apiobject, which declares the available parameters this fragment can consume, and optionally specify default values. It is expected that the policy tooling will use this object to validate the provided parameters when generating a policy with fragments.Example fragment using this feature:
Example policy importing this fragment:
If multiple fragment parameters are specified, all combinations of values are considered "allowed", and therefore the fragment injection is repeated, passing in different parameter combinations, one for each combination. This means that if a fragment defines, for example:
and we have two fragment import statement for this fragment, e.g.:
Then the container can be started with either SOME_KEY=value1 or SOME_KEY=value2.