FE-1341: Give experiment compute backends one interface - #9178
Draft
kube wants to merge 1 commit into
Draft
Conversation
Choosing a backend was hardcoded in ExperimentsProvider. The runtime half of swappability already worked — a backend produces a MonteCarloExperiment and consumers drive one with no branching — so this adds only the missing part: asking a backend whether it can run a net, and choosing between backends when one declines. A request is plain serializable data; anything describing how to compute belongs to the backend's construction or to per-call options. Refusal is a value carrying structured blockers with a code, an item id and an origin, so a UI can attribute a problem to the item that caused it. Assessment settles the net without acquiring a device or worker pool and hands back an instantiate() closure. Pure refactor: no user-visible behaviour changes. Registrations carry a deferred load so a heavy backend need not enter the bundle.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🌟 What is the purpose of this PR?
Choosing a compute backend is about to stop being a single choice. Today
ExperimentsProviderhardcodes it — try one, and if it declines, use the other and put the reason in a notification — which works for exactly two backends and has to be edited to gain a third.This introduces the contract both backends satisfy, so the WebGPU backend stacked above plugs into a seam rather than rewiring the provider.
Pure refactor: no user-visible behaviour changes, and no changeset.
🔗 Related links
🚫 Blocked by
🔍 What does this change?
The runtime half of swappability already worked and is untouched: a backend produces a
MonteCarloExperiment, and consumers drive one with no branching. What was missing is asking a backend whether it can run a net, and choosing when one declines.A new
@hashintel/petrinaut-core/experimentsentry point adds:ExperimentRequest— what to compute, as plain serializable data. Anything describing how belongs to the backend's construction or to per-call options. That rule is what stops this becoming a union of every backend's knobs, and it is whysignaland note callbacks are not on it — for a future out-of-process backend the request is the request body.ExperimentAssessment— refusal as a value, not an exception, because a subset engine declining a net is ordinary operation. Blockers carry acode, an optionalitemId, and anorigin(model/configuration/environment/capacity) so a UI can attribute a problem to the transition that caused it.capacityis separate fromenvironmentbecause a device momentarily out of memory warrants "use fewer runs", not "hide the option".ExperimentBackend— two phases. Assessment settles the net and configuration without acquiring a device or worker pool, so assessing while the user edits is cheap, and hands back aninstantiate()closure carrying that work forward.selectExperimentBackend— walks registrations in preference order, records why each declined, and treats a failed instantiation as a refusal so a runnable net still falls back when a resource cannot be acquired.flowchart TD R[registrations, preference order] --> L[load backend] L --> A{assess request} A -->|blockers| N[record refusal] --> R A -->|eligible| I{instantiate} I -->|environment / capacity| N I -->|handle| D[run, recording which backend]Registrations carry a deferred
load, so a heavy backend can be registered without pulling its implementation into a bundle that never uses it.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
Only one backend is registered at this point, so
assessnever refuses and the blocker machinery is unexercised until the WebGPU backend lands above. That is the cost of introducing the seam before its second implementation; the alternative was a GPU PR that both adds a backend and rewires the provider.Resource sharing across experiments — a reused worker pool, a shared
GPUDevice— was considered and deliberately left out. Worker reuse depends on the worker fully resetting frame and metric state, and if it does not, results stay plausible rather than visibly wrong.🐾 Next steps
Providing backends through a React context, so an embedder can supply its own. The contract is shaped for it: a provider builds the backends its environment supports and publishes the registration list.
🛡 What tests cover this?
select-experiment-backend.test.tscovers the walk: first-accepting wins without loading later registrations, fallback records the refusal, a failed instantiation falls through, blockers are ordered by actionability, an unavailable backend is skipped without assessment, a backend whose module fails to load does not sink the experiment, and requests are built once per distinct artifact requirement.❓ How to test this?