Skip to content

Parameter Manager: Types, Locks and a new GUI - #159

Open
marcosfrenkel wants to merge 108 commits into
toolsforexperiments:masterfrom
marcosfrenkel:marcosfrenkel/new-param-manager
Open

marcosfrenkel wants to merge 108 commits into
toolsforexperiments:masterfrom
marcosfrenkel:marcosfrenkel/new-param-manager

Conversation

@marcosfrenkel

@marcosfrenkel marcosfrenkel commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Adds Types and Locks to the Parameter Manager and rebuilds its GUI around them. Everything works from a Python client and from the Qt GUI, and open GUIs update live when another client changes something.

Why

  • Multi-qubit devices repeat the same structure per qubit. Today each copy is edited by hand and drifts apart. Types make the structure explicit and keep the copies complete.
  • Shared settings (an LO frequency, a readout bandwidth) are duplicated across qubits. Locks make one parameter authoritative without copying values around.
  • Measurement code needs these as well as the GUI, so they are instrument methods first and widgets second.

What's new

Types

A Type is a named shape: relative parameter paths with a default and a unit, plus Nested Types at named submodules (qubit requires a readout at readout). Any submodule carrying that shape with the right units is an Instance. Membership is structural and recomputed on demand, never stored (ADR 0001).

  • add_type, remove_type, add_type_parameter, remove_type_parameter, add_nested_type, remove_nested_type, instances_of, types_of, add_instance.
  • Adding an entry to a Type writes it into every Instance. add_instance builds a new Instance from the Type's defaults and refuses up front on a unit conflict.

Locks

A Lock makes one parameter (the Follower) read another (its Target). While locked, the Follower answers get with the Target's value and refuses set. Unlocked, it acts as a plain parameter but remembers its Target.

  • lock, unlock, relock, toggle_lock, remove_lock, get_lock, list_locks.
  • Values are pulled on get; nothing is ever pushed into a Follower (ADR 0002). This lives in a ManagedParameter subclass, so the Server's call path and the instrument mutex are unchanged. Snapshots of a locked Follower report the Target's value, so measurement metadata isn't stale.
  • Locks can chain but never cycle. Targets must be in the same Parameter Manager.
  • Deleting a Target removes the Locks pointing at it; their Followers become plain parameters.

Type Locks and _globals

lock_type_parameter / unlock_type_parameter declare a Lock on a Type entry, so every Instance follows one Target. By default that Target is a parameter under _globals. _globals is never matched as an Instance.

Broadcasts: the Broadcaster contract

Until now the Server only broadcast what it could see itself: parameter sets, and calls literally named add_parameter / remove_parameter. A new opt-in mixin, Broadcaster (base.py), lets an instrument emit its own Broadcasts. The Server registers itself as a sink when such an instrument joins the Station (ADR 0003). Instruments without the mixin are untouched.

The Parameter Manager is the first Broadcaster. It emits:

  • pm-lock-update (payload PMLockBluePrint, or None when a Lock is removed)
  • pm-type-update (payload PMTypeBluePrint, or None when a Type is removed)
  • parameter-creation for parameters it creates as side effects (Type edits, add_instance, Type Lock declarations)

Wire format, socket and topic are the same as before, so existing subscribers parse these unchanged. Broadcast action names are now constants in blueprints.py.

Profile files: version 2

toFile writes a versioned document (version: 2): a parameters section (each entry holds its own value, its unit and, if it has one, its lock) and a types section. fromFile reads both version 2 and the old unversioned files. Before changing anything it validates the whole document: if Lock or Type Lock Targets are missing, it raises one ValueError listing all of them and leaves the state untouched. Switching profiles now clears Types and Locks too.

GUI

The Parameter Manager GUI moved out of gui/instruments.py into the new gui/parameter_manager/ package (logic.py is Qt-light state and helpers, widget.py and panels.py hold the widgets).

  • Parameters tab: rows tinted by their claiming Type, with gutter bands that show Instance boundaries. A "locked to" column with a lock toggle, a context menu, and an arm strip for picking a Target by clicking a row.
  • Locks panel (Ctrl+Shift+L): every Lock in one table, with resizable columns.
  • Types tab: a list of Types with Instance counts, the selected Type's entries, and its Instances.
  • Deleting a parameter that is a Target asks for confirmation first.
  • Tints follow the application theme, with a palette for dark themes too.

The generic parameter GUI (gui/parameters.py) gained column and editor hooks, which the Parameter Manager uses.

Smaller fixes along the way

  • _newOrDeleteParameterDetection no longer fails on add_parameter calls that omit initial_value or unit.
  • The instrumentserver-param-manager launcher passes the broadcast port into the GUI, so its live updates work.
  • QLogHandler is a plain logging.Handler now, so closing the app no longer prints "wrapped C/C++ object of type QLogHandler has been deleted". A dead handler also no longer makes the logger skip the handler after it.
  • Tests pick free ports per run instead of fixed ones, so parallel runs don't collide.

Docs

  • User Guide: docs/user_guide/parameter_manager.md rewritten (Types, Locks, Type Locks and Globals, profiles and files, the GUI, use from measurement code).
  • Technical Guide: docs/technical_guide/broadcasts.md covers what triggers a Broadcast, the wire format, the Broadcaster contract, and the Parameter Manager's actions.
  • Both pages are verified by scripts under test/docs_verification/ that run every example against a live Server.
  • Design records: CONTEXT.md (glossary) and docs/adr/0001–0003.

Compatibility

  • Old unversioned profile files still load. Files this branch writes are version 2, which older instrumentserver releases can't read.
  • Instruments that don't use Broadcaster behave exactly as before.

Testing

About 10,000 lines of new tests: test_pm_locks.py, test_pm_types.py, test_pm_persistence.py, test_pm_gui.py, test_broadcaster.py, test_log_widget.py.

With QT_QPA_PLATFORM=offscreen and the venv on PATH, the full suite passes (556 tests). Run with the native macOS style, two layout tests in test_pm_gui.py fail because they expect offscreen Qt's layout spacing (the tree-height one and the Locks panel column-resize one). That's a test issue, not a GUI bug.

Reading this PR

It's large (~26k lines added, most of them tests and docs). The history is meant to be read commit by commit: each implementation commit starts with its plan task number (0.1, 1.2, …), and review fixes are separate commits. PLAN_parameter_manager_redesign.md explains each task and records the design decisions; start with the ADRs and CONTEXT.md.

🤖 Generated with Claude Code

marcosfrenkel and others added 30 commits September 23, 2026 16:06
- Adds .agents/ roles (coder, three reviewer types) and the orchestrate-plan skill so plan tasks can run through a coder + six reviewer agents driven by opencode/Orca.
- Records the Types/Locks design work: PLAN_parameter_manager_redesign.md, three ADRs, and updated CONTEXT.md glossary entries (Broadcaster, Type, Lock, etc).
- Adds PLAN_docs_refactor.md tracking the documentation rewrite, plus TEST_AUDIT.md and TODO_type_cleanup.md to carry forward gaps found along the way.
- opencode.json wires up the coder/reviewer agent permissions matching the roster.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Swap `lumen/qwen3-coder-next` for `lumen/qwen3.8-27b` across reviewer/test-reviewer/plan-checker roles
- Add `wait-event.sh` so orchestrator polls for messages/permission prompts instead of sitting in a long blocking `check --wait`, since permission prompts don't arrive as messages
- Allow `cd`, `pwd`, `git branch --show-current`, `lsof`, and `sed -n` for opencode agents
- Add plan task 0.0 for per-run test ports (fixes parallel test runs colliding on fixed ports 5555/5556/5599)
…e in workers

- opencode.json: allow read-only shell tools for all agents (find without
  -delete/-exec, git grep/ls-files, ps, lsof, sort, diff, jq, ...); allow
  sed -i / perl -pi for the coder, deny them for reviewers.
- Role files: 'Reading code' section pointing at .venv site-packages.
- ROSTER.md: launch workers with PYTHONDONTWRITEBYTECODE=1; permission lists.
- Plan: 0.0 acceptance uses git grep; Testing conventions use server_port (D27).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…adcast port to the parameter manager GUI launcher
…al site in server, gui and client application
… reports

- New historian role (.agents/roles/historian.md, bin/historian-claude.sh):
  a Claude Opus session that turns each task's reviews and decision log into
  one section of HISTORY_<plan>.md. Skill Step 7 runs it at the end of every
  task; the orchestrator's end-of-task commit is now 'T: history'.
- HISTORY_parameter_manager_redesign.md: backfilled sections for 0.1, 0.0,
  0.2, 0.3, 0.4, 0.5 (in the order they were done).
- orchestration/ is git-ignored and untracked; the raw files stay on disk.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The deepseek-v4-flash reviewers stalled 11 times across 0.3-0.5 (provider
upstream errors, garbled output). Their three slots are now reviewer-glm,
test-reviewer-glm and plan-checker-glm on lumen/glm-5.3-flash.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n-path tests for all Lock methods, relock missing-Target test
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
marcosfrenkel and others added 28 commits September 28, 2026 14:36
…ailure and empty-submodule tests, nested-instance matching test, stripped unit
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…llower dialog line and default button, removalDialog reset, parameter-call recompute
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… claim, GUI wording, profiles setup and legacy flow, script coverage
…he Target deletion and make capture_one_broadcast return the post-settle messages
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…very Type-editing method, double-registration and wording fixes
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… final pass, ADR toctree warnings fixed, User Guide forward reference reworded
…o audit rows for the 1.2/1.3 loose ends, broadcasts bullet renamed to its section title
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
gui/instruments.py is back to the generic parameter and method display
and GenericInstrument. The Parameter Manager GUI now lives in
gui/parameter_manager/: logic.py (no widgets: claims, tints, Lock and
Types rows, PMState), panels.py (gutter delegate, arm strip, Locks panel,
Types tab) and widget.py (ParameterManagerGui, its model, tree view,
delete delegate, create form and profiles box). The code is moved line
for line; only imports, section banners and six cross-module docstring
references changed.

instruments.py keeps a lazy module __getattr__ so station configs naming
instrumentserver.gui.instruments.ParameterManagerGui (or PMState) keep
loading. In-repo configs, docs and tests use the new path.

The lockChanged/typeChanged signals and the pm-lock-update/pm-type-update
branches move from the generic ModelParameters to ModelParameterManager;
a plain ModelParameters now ignores those Broadcasts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ModelParameters now asks headerLabels() and rowItems() for its columns
and row cells instead of pinning three columns after loading. The
Parameter Manager model extends both, so its copied insertItemTo, the
post-load column widening, the _ensure_extra_items walk and the two
fallbacks in the tint and Lock passes that created missing cells are gone.

ParameterDelegate.createEditor builds the editor through
makeParameterWidget(); the Parameter Manager's delete delegate overrides
only that instead of copying createEditor. ParametersTreeView takes its
delegate from delegateClass and calls setupColumns() before opening the
persistent editors; ParameterManagerTreeView now derives from it and
drops its copy of onItemNewValue.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ModelParameterManager no longer scans the tree before and after every
value Broadcast to tell whether a row was added: it listens to its own
newItem while a Broadcast is handled and emits structureChanged once when
rows were added or a parameter was removed.

Leftovers from the split:
- lock_row_paths is public; widget.py no longer imports a private name.
- LockableParameterWidget carries the row's lock button as a declared
  attribute instead of one set on ParameterWidget from outside.
- ParameterManagerTreeView.setupColumns drops its column-count checks;
  the view is always built over a ModelParameterManager (the one test
  that used a plain ModelParameters now uses the PM model).
- The package exports only ParameterManagerGui, PMState and
  ModelParameterManager; the tests import the rest from the submodules.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ParameterManagerGui (about 1,000 lines) owned the tints, the Lock column
and buttons, the arm strip's target pick, the Locks panel handlers, the
Types pane handlers and about 50 signal connections. It now builds the
widgets and hands the behaviour to two controllers that wire their own
signals:

- LocksController: the Lock column, lock buttons and read-only values,
  the lock/unlock actions and shortcuts, the arm strip (armed_follower,
  armed_type_lock) and the Locks panel.
- TypesController: the tints and gutter bands, and the Types tab actions.

The methods moved unchanged apart from reaching the GUI's widgets through
self.gui; the set of signal connections is the same, and the Types
controller still connects before the Locks one so a structural change
repaints the tints before the Lock pass. The model's {path: unit} walk
moved to ModelParameterManager.parameter_units(), which both controllers
use. Tests reach the arm state and methods through gui.locksController.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Moving the tree into the Locks splitter re-inserted it without a stretch
factor, so the tree and the add-parameter strip split the spare height
and a band of empty space opened under the tree. The splitter now takes
stretch 1 and the strip a fixed height.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Server broadcasts a parameter-call for every get, and the model hands
it on as a new value. With the Locks panel open, _on_item_new_value
answered that with a fresh get of the same path (a locked Follower's
label), which broadcast again: thousands of requests a second while
nothing happened. The slot now paints the tree's Followers and the
panel's rows with the Broadcast's value (a locked Follower reads its
Target's value, D3) through the new LocksPanel.show_value; a Lock change
still re-reads its rows once, and the Broadcasts that causes only repaint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Qt deletes a row's persistent editor when the filter or the trash toggle
hides the row, but ParameterDelegate.parameters kept the dead widget.
The Parameter Manager re-applies the Locks on every filter change, so a
filter that hid a locked row ("LO") crashed with "wrapped C/C++ object of
type QPushButton has been deleted". The delegate now drops the entry on
the editor's destroyed signal (unless a newer editor replaced it), and
onItemNewValue skips a row that has no editor.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The value and buttons columns were fixed and the locks column stretched,
so no header edge could be dragged and the Target's value editor was
squeezed until its number was unreadable. The locks and buttons columns
are now interactive and the value column takes the rest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
QLogHandler was a QObject as well as a logging.Handler. Qt deletes it
with the log widget, and logging.shutdown then visits it at interpreter
exit, printing "wrapped C/C++ object of type QLogHandler has been
deleted". The handler is now a plain logging.Handler; the text widget,
the signal and the slot live in a _LogBridge parented to the widget. The
handler reaches the signal through the bridge on each emit, so a record
after the widget is gone raises the RuntimeError the handler already
catches (a cached bound signal would segfault instead).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The tree's row tints and gutter bands were light-only and made the
theme's light text hard to read on a dark desktop. A second palette with
the same five hues is picked when the application palette is dark, and
the GUI re-tints when the palette switches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a log widget is gone, its handler detached itself with
removeHandler while the logger was looping over that same list, so the
handler after it missed the record. It now replaces the list instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@marcosfrenkel marcosfrenkel changed the title Marcosfrenkel/new param manager Parameter Manager: Types, Locks and a new GUI Oct 2, 2026
ruff format the files the branch touched and sort one import block.
mypy: Qt's stubs return Optional from header(), selectionModel(),
invisibleRootItem(), style() and addAction(), so those get asserts; the
model's rows are cast to ItemBase where their name and element are read;
paint and changeEvent take Optional arguments like their base classes.
In params.py the Type Lock loop now keeps only the entries that have a
Target, so it carries a str rather than an Optional.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant