Skip to content

feat(fp): expose global plasma scalars (Ip, stored energy, powers, volume) in fp_state_t - #233

Merged
k-yoshimi merged 8 commits into
k-yoshimi:developfrom
HengyuLi-Ozaki-lab:pr/fp-state-global-scalars
Oct 20, 2026
Merged

k-yoshimi merged 8 commits into
k-yoshimi:developfrom
HengyuLi-Ozaki-lab:pr/fp-state-global-scalars

Conversation

@HengyuLi-Ozaki-lab

Copy link
Copy Markdown
Collaborator

Summary

Adds the seven geometry-dependent global plasma scalars that FP already computes to fp_state_t, exposed as a TR-style scalars block:

field unit Fortran source
TOTAL_IP MA Σ_NSA PIT(NSA,NTG1) — fpsave.f90:110,146,286, printed :311
STORED_ENERGY MJ Σ_NSA PWT(NSA,NTG1) — fpsave.f90:112, printed in the W column :252
COLLISION_POWER MW Σ_NSA PPCT(NSA,NTG1) — fpsave.f90:114,278, printed :307
ABSORPTION_POWER MW Σ_NSA PPWT(NSA,NTG1) — fpsave.f90:115,277, printed :305
ABSORPTION_WR MW Σ_NSA PWRT(NSA,NTG1) — fpsave.f90:284, same print line
ABSORPTION_WM MW Σ_NSA PWMT(NSA,NTG1) — fpsave.f90:285, same print line
PLASMA_VOLUME m³ TVOLR — fpprep.f90:231-234, banner :237

All seven were already available as fpcomm module data (PIT/PWT/PPCT/PPWT/PWRT/PWMT are (NSAMAX, NTG1M) allocatables; TVOLR is a scalar). None is computed at print time, so all are retrievable after fp_run. The MW/MA conversions are the ones already applied in the moment integrators (fpsave.f90:1310, :1377, :1509, :1510).

Motivation

FP's get_state currently returns only the six per-species profile arrays and the mesh dimensions. Those six are local per-unit-volume moments: with MODELD=0 and no wave or current-drive model, each radial cell is an independent 0-D collisional relaxation depending on local n, T, Z and E — not on B, R or ε.

The practical consequence: loading a completely different equilibrium changes nothing observable. Running fp_iter01 against an analytic equilibrium versus against an eq.bin written by eq's save tool with substantially perturbed geometry (RR 3.0→3.6, RA 1.0→1.2, BB 3.0→4.5, RIP 1.0→2.0, RKAP 1.5→1.8) produces bit-identical RNT/RWT/RTT/RJT/RPCT/RPWT.

The equilibrium is being consumed — eqgetb writes the loaded geometry back into FP, the mesh metrics move (rsrhon 0.805→1.0264, bpm 0.3506→0.1829, ql 2.2961→7.0134), and the global integrals scale with the volume ratio — but it enters only through the volume element and the mesh metrics, and fp_state_t exposed neither. So an MCP/library client driving the MODELG=3 equilibrium-load path had no way to tell whether the file had been used at all.

(Ruled out as a stale-state artifact: changing PTPR/PTPP, PN or NTMAX does move the arrays. Two further attempts to surface a difference — a strong loop voltage driving RJT, and radial transport with MODELD=1 — were also bit-identical, with a control confirming the perturbed eq.bin had been loaded in that arm.)

Why a scalars block

trlib's TrState.to_dict() already returns a top-level "scalars" key; fplib's did not. Beyond exposing the new values, this removes that shape inconsistency — downstream code that reads state["scalars"] generically (for example steady-state detection that looks up a named scalar) silently got None for FP.

Verification

Rebuilt with the repo's current make.header (-fbounds-check -fcheck=all, unmodified).

1 — values agree with the Fortran's own output. Each scalar was compared against the WRITE(6,...) lines of the same run at full precision: TVOLR 0.284245E+02, W 5.3683E+00, collision -2.1308E-01, IP 1.4970E-13, absorption 0.0000E+00. This simultaneously validates the unit conversions and the struct layout — a layout error would read garbage. A supplementary probe with E0=0.05 drove TOTAL_IP to 45.24848, matching the printed 4.5248E+01.

2 — the equilibrium is now observable, and the ratios reproduce what the Fortran diagnostics independently showed:

scalar analytic equilibrium loaded eq.bin (perturbed) ratio
PLASMA_VOLUME 28.42446 53.12014 1.8688
STORED_ENERGY 5.368302 10.33949 1.9260
COLLISION_POWER −0.2130797 −0.4073574 1.9118

The six profile arrays remain bit-identical between those two runs — i.e. the underlying physics is unchanged and it is the new scalars that make the difference visible.

3 — no regression. State captured with the pre-change .so and with the rebuilt one is bit-identical across all six arrays, the five integer dimensions and TIMEFP. Purely additive.

4 — initialisation safety. After init without run, all seven scalars read 0 (the g_prepared guard prevents an uninitialised TVOLR from leaking); after run, they are populated. Filling is bounds-guarded with MIN(NSAMAX, SIZE(PIT,1)) / MIN(NTG1, SIZE(PIT,2)).

5 — tests. fplib 47 passed (was 45), fp_mcp 46 passed (was 45). New coverage asserts the seven fields are appended last in FpStateC._fields_ and are all c_double, that from_c/to_dict carry them, and that the MCP STATE_SCHEMA stays in sync with SCALAR_FIELDS.

6 — determinism. 5/5 fresh runs of each arm, byte-identical outputs.

Honest caveats

  • ABSORPTION_POWER / ABSORPTION_WR / ABSORPTION_WM are verified only at zero. I could not drive them non-zero through the C ABI: MODEL_WAVE=1 with PABS_LH=1.0 still yields 0, and the LH/EC resonance-mesh parameters (DELNPR, NPRMAX, …) are not in fp_param_registry.f90, so they are unreachable from this path. The summation follows exactly the same pattern as the two power fields that are verified against non-trivial values, but a reviewer with a working wave configuration should sanity-check them.
  • ABI change. The seven fields are appended at the end of fp_state_t, so libfpapi.so must be rebuilt; python/fplib/_ffi.py is updated in lockstep.
  • test_equivalence.py::test_iter01 fails on this branch — pre-existing, not introduced here. Verified by stashing the entire change, rebuilding, and re-running: the same failure appears with 37 mismatches, all in profile[*].RPCT at rel_err ~1e-9–1e-10. Structurally it cannot be affected either: _to_baseline_shape() reads only the six arrays, the integer dimensions and TIMEFP, never the new scalars block.

Files

fp/fp_state.f90, fp/fp_api.f90, fp/fp_api.h, python/fplib/_ffi.py, python/fplib/state.py, python/mcp-servers/fp_mcp/server.py, plus tests in python/fplib/tests/ and python/mcp-servers/fp_mcp/tests/.

Independent of the two companion fp_mcp PRs (#231 set_param_str, #232 stdout guard); this one may need a trivial rebase if #231 lands first, since both touch fp_mcp/server.py in different places.

🤖 Generated with Claude Code

…lume) in fp_state_t

Motivation
----------
`fp_get_state` returned only six per-unit-volume profile arrays
(RNT/RWT/RTT/RJT/RPCT/RPWT) plus mesh sizes. In the common configuration
(MODELD=0, no wave model, no current drive) those moments are computed
grid-point-locally, so the equilibrium enters the physics *only* through
the volume element VOLR and the grid metric -- neither of which was in
the state. Measured on the fp_iter01 fixture, swapping the analytic
equilibrium for an EQ-produced eq.bin with a very different geometry
(RR 3.0->3.6, RA 1.0->1.2, BB 3.0->4.5, RIP 2.0, RKAP 1.8) leaves all
six arrays **bit-for-bit identical**, while the underlying plasma volume
nearly doubles. EQ->FP handoff therefore worked in the library but was a
silent no-op to any MCP client.

The quantities that *do* respond were already computed by FPWRTGLB --
they were just WRITE(6,...)'d and thrown away. This commit routes them
into the state struct.

New fields (all species-summed at the latest NTG1 sample)
--------------------------------------------------------
  TOTAL_IP         [MA]  sum PIT;  PIT = sum_NR RJS*VOLR/(2*pi*RR).
                         RJS carries the *1.D-6 A->MA factor at
                         fp/fpsave.f90:1310; printed at fpsave.f90:311.
  STORED_ENERGY    [MJ]  sum PWT;  PWT = sum_NR RWS*VOLR.
                         RWS is *1.D-6 J->MJ at fp/fpsave.f90:1377;
                         printed as the W column of fpsave.f90:252.
  COLLISION_POWER  [MW]  sum PPCT; RPCS *1.D-6 at fpsave.f90:1509,
                         printed at fpsave.f90:307.
  ABSORPTION_POWER [MW]  sum PPWT; RPWS *1.D-6 at fpsave.f90:1510,
                         printed at fpsave.f90:305.
  ABSORPTION_WR    [MW]  sum PWRT -- the "WR:" term of fpsave.f90:305.
  ABSORPTION_WM    [MW]  sum PWMT -- the "WM:" term of fpsave.f90:305.
  PLASMA_VOLUME    [m^3] TVOLR = sum_NR VOLR, fp/fpprep.f90:231-234,
                         printed in the DEVICE banner at fpprep.f90:237.

All seven are zero until the first run: TVOLR and the P*T(NSA,NTG1)
accumulators only exist after fp_prep, and only carry a sample once
FPSGLB has bumped NTG1 (fp/fploop.f90:200). fp_get_state guards on
g_prepared + NTG1 >= 1 + ALLOCATED so it never exposes uninitialised
memory.

Why a `scalars` block
---------------------
The Python wrapper groups them under a top-level `scalars` key, matching
`trlib.state.TrState.to_dict()`. FP was the odd one out: consumers that
branch on `state.get("scalars")` (e.g. task-web's mcp_bridge
steady-state detection) silently saw None for every fp state. Same shape
now means the same code path works for both modules.

ABI
---
The 7 doubles are APPENDED to fp_state_t, so every pre-existing member
keeps its offset. libfpapi.so MUST be rebuilt (`make -C fp libfpapi.so`)
-- a stale .so read through the longer ctypes layout returns garbage in
the trailing members. Three declarations are kept in lockstep:
fp/fp_state.f90, fp/fp_api.h, python/fplib/_ffi.py::FpStateC; a new test
asserts the scalars are the last 7 fields and all c_double.

Verification (fp_iter01 fixture, NTMAX=2)
-----------------------------------------
                    G1 analytic     G2b eq.bin      ratio
  PLASMA_VOLUME       28.42446       53.12014      1.8688
  STORED_ENERGY        5.368302      10.33949      1.9260
  COLLISION_POWER     -0.2130797     -0.4073574    1.9118
  TOTAL_IP             1.497e-13      1.635e-13    (noise; no E-field)
  ABSORPTION_*         0              0            (no wave model)

Every value agrees with the corresponding Fortran WRITE(6,...) line of
the same run to full printed precision, which is the correctness check
for both the units and the struct layout. TOTAL_IP was separately
exercised with E0=0.05: state 45.24848 vs printed 4.5248E+01.

The six profile arrays are bit-for-bit unchanged versus a run against
the pre-change .so, confirming this is purely additive and does not
touch the computation path.

Tests: fplib 47 passed, fp_mcp 46 passed. The single
test_equivalence::test_iter01 failure (37 profile[*].RPCT mismatches at
1e-10) is PRE-EXISTING -- reproduced identically on a pristine rebuild
of HEAD, and _to_baseline_shape() does not read the new `scalars` key.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@HengyuLi-Ozaki-lab

Copy link
Copy Markdown
Collaborator Author

This has been green since 2026-08-06 and has had no review yet. Could one of you look at it when you have time? It is
small and self-contained (9 files, +228/-14: the global plasma scalars in fp_state_t) and the FP work we are doing downstream builds on it. It does not depend
on #226, and a trial merge of #226, #231, #232, #233 and #240 together is clean.

@k-yoshimi k-yoshimi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 92a0163d (based on current develop tip). Approving; one follow-up is filed on the maintainer side (below).

Verified locally (macOS arm64, gfortran 15, rebuilt fp/libfpapi.so from the PR head, pytest 9.1.1 --forked --timeout=120 --timeout-method=signal, strict-subTest override from #243 active):

  • python/fplib + python/mcp-servers/fp_mcp: 88 passed, 3 skipped (Linux-canonical equivalence).
  • Live probe (NPMAX=NTHMAX=20, run(2), unit 6 captured at fd level): the state's seven scalars equal the last FPWRTGLB block to every printed digit — TOTAL_IP 3.3668E-14, COLLISION_POWER 3.4465E-12, STORED_ENERGY 3.7137E-01 (the W column), PLASMA_VOLUME 3.55306 (the TVOLR banner). Earlier NTG1 samples differ, confirming the "latest NTG1" semantics is what is exposed.
  • Array shapes cross-checked: PIT/PWT/PPCT/PPWT/PWRT/PWMT are allocated (NSAMAX, NTG1M) (fp/fpcomm.f90:1204-1217), so SIZE(…,1) for species and SIZE(…,2) for the sample index are the right guards; TVOLR is a module scalar; the g_prepared / NTG1 >= 1 / ALLOCATED guards keep the zeros honest before the first run.
  • Layout: the 7 doubles are appended after RPWT in all three declarations (fp_state.f90, fp_api.h, _ffi.py), and test_global_scalars_are_appended_last pins that.

Follow-up (maintainer side, not asked of this PR): the 1e-10 binary-vs-library equivalence gate does not see these scalars yet. fp/fpregress.f90 dumps only TIMEFP, extract_fp_metrics.py's SCALAR_KEYS and the _flatten in python/fplib/tests/test_equivalence.py mirror that, and the fp_* baselines would need regeneration on canonical Linux (the regen workflow covers tr/eq/tot only). That is the same "state field added without the regress mirror" class we hit with AJRFT in TR (#187 → #190/#191), so I am filing it as an issue and will do the fpregress + extractor + baseline regeneration myself after this merges. Nothing in this PR blocks on it: compare_metrics.py compares the scalars the Python side emits, and the Python equivalence flatten still emits TIMEFP only, so CI stays green until the mirror lands with its baselines.

@k-yoshimi

Copy link
Copy Markdown
Owner

Independent Codex review (second reviewer, read-only). One concrete regression needs a fix in this PR; the ABI point becomes a repo-wide follow-up issue.

MED — totlib pipeline output changes silently (fix in this PR, please). python/totlib/pipeline.py::_state_to_scalars (lines 149-172) returns dict(state.scalars) as soon as the state has a scalars dict, and only falls back to the numeric top-level attributes for modules without one — that fallback is what FP has been using. With this PR, FP pipeline results (pipeline.py:558 and :650, including partial results) switch from {nrmax, nsamax, npmax, nthmax, ntg2, timefp} to the seven new globals and lose timefp. The existing pipeline tests use mocks that already carry .scalars, so nothing caught it. Suggested fix: in _state_to_scalars, when the state has scalars, still merge in timefp (and the int dims) for FP — or, simpler and uniform, always start from the numeric top-level attributes and update with state.scalars — plus one regression test that feeds a real FpState through it.

Docs to update in this PR: docs/sphinx/modules/fp/en/state.md (lines 19-20 and the table at 74) and the ja twin say state.scalars holds only timefp and that FP has 1 scalar; python/fplib/README.md and docs/fp-library/architecture.md still list the old struct only.

HIGH per Codex, filed as #246 (repo-wide, not a blocker here). Appending members keeps the old offsets but not the allocation size: an old caller with a new .so gets 56 bytes written past its struct during the unconditional zeroing, and a new caller with an old .so reads seven untouched zeros that look like real values (the scalars block then looks valid to an MCP client). The repo's fp_state_t/tr_state_t/… ABI has no size or version query at all, so this is not specific to this PR — it is the same for every state-struct growth we have done (AJRFT in TR, for one). I will track a size-negotiated getter / ABI-version symbol across modules in #246; for this PR the existing rule "rebuild libfpapi.so in the same commit" stands.

LOW (optional): STATE_SCHEMA["scalars"] is required but none of its seven members is (required: list(SCALAR_FIELDS) would make the contract explicit); test_size_matches_header_math checks ctypes arithmetic only, not a C sizeof — covered by #246.

Codex confirmed, consistent with my local probe: field order/types match across the three declarations; NTG1 cannot exceed the accumulators' second dimension; the MA/MJ/MW units match the 1.D-6 conversions in fpsave.f90; TVOLR is valid after fp_prep even with zero steps; libtotapi_mono.so needs no tot_state_t change.

Once the pipeline fix + docs are pushed I will re-approve.

… the FP scalars

`_state_to_scalars` returned a copy of `state.scalars` for any state that
has that dict, and the numeric top-level attributes otherwise. FpState had
no dict, so an FP step's scalars were {nrmax, nsamax, npmax, nthmax, ntg2,
timefp}. With the `scalars` dict this branch gives FpState, they became
the seven globals alone and `timefp` was gone from pipeline results. The
pipeline tests did not see it: their mocks already carried `.scalars`.

For an FpState the function now returns the top-level numbers together
with the dict. An FpState is told by its class's name and module, so
nothing is imported for the test. Other modules are untouched: a state
with a `scalars` dict still gives exactly that dict (the uniform
alternative, top-level numbers for every module, would add nt, nrmax, nsmax
and the like to the results of tr, eq, wr and wrx, and `call_count` to a
MagicMock's).

Tests: a real FpState through `_state_to_scalars` (13 keys, a copy); an
FpState class reached under another package name, and a class that only
shares the name; a state of another module with top-level counts keeps
only its dict; and one FP step through `run_pipeline` on the real library,
compared key by key with `get_state()`. The first, the second and the
last fail without the fix.

Documentation that still described the state before the scalars: the FP
state page (en, ja) and its comparison table, the same row in the wr state
pages, python/fplib/README.md, docs/fp-library/architecture.md. They say
what was measured on this build: six of the seven are sums over the
species at the latest NTG1 sample and PLASMA_VOLUME is TVOLR; all seven
are zero until the first run, and again after a set_param until the next
run. The state schema of fp_mcp lists the seven scalars as required, with
its test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@k-yoshimi

Copy link
Copy Markdown
Owner

Heads-up for the rebase: the #245 fix (branch fix/fp-get-state-extents-245, PR to follow #242) changes the profile-copy loop in fp_api_get_state to nr_alloc = MIN(NRMAX, SIZE(RNT,1)), nsa_alloc = MIN(NSAMAX, SIZE(RNT,2)), ntg_last = MIN(MAX(NTG2,1), SIZE(RNT,3)) and reports the copied extents in state%nrmax/nsamax. Your scalar block is inserted after that loop's END IF, so it should 3-way merge cleanly; it already bounds the P*T sums by SIZE(PIT, ...), which is the same pattern. Just make sure nothing in the inserted block loops to the bare NRMAX / NSAMAX.

@HengyuLi-Ozaki-lab

Copy link
Copy Markdown
Collaborator Author

Pushed e84d336b.

The pipeline regression. Reproduced first: one FP step through run_pipeline on the real library returned the seven globals and no timefp. _state_to_scalars now gives, for an FpState, its numeric top-level attributes together with its scalars dict: 13 keys, the six it had before and the seven new ones.

I took the FP-only form rather than the uniform one, because the uniform one changes every other module's pipeline results: TrState, EqState, WrState and WrxState have top-level counts (nt, nrmax, nsmax, nine dimensions for EQ, modelg and mdlwrq for WRX) that would join their scalars, and the existing mock-based test gains a call_count. An FpState is told by its class's name and module, so totlib imports nothing for it. If you prefer the uniform form I will change it.

Tests: a real FpState through the function; an FpState reached under another package name; a state of another module with top-level counts keeps exactly its dict; one FP step through run_pipeline on libfpapi.so, compared key by key with get_state(). The first, the second and the last fail without the fix.

Documentation. docs/sphinx/modules/fp/{en,ja}/state.md (the scalar section and the comparison table), python/fplib/README.md, docs/fp-library/architecture.md; and the same "1 (timefp)" cell in docs/sphinx/modules/wr/{en,ja}/state.md. Two things the new text says that the comment in python/fplib/state.py does not, both measured on this build: six of the seven are sums over the species at the latest NTG1 sample while PLASMA_VOLUME is TVOLR; and all seven are zero until the first run() (run(0) is enough) and again after a set_param, until the next run(), since fp_set_param clears g_prepared.

LOW. STATE_SCHEMA["scalars"] has required: list(SCALAR_FIELDS), with its test.

Your heads-up about #248. The scalar block does not loop to the bare counts: ntg1_last = MIN(NTG1, SIZE(PIT, 2)) and DO nsa = 1, MIN(NSAMAX, SIZE(PIT, 1)). A trial merge of this branch with #248 (f7d8c2da) on develop has one conflict, in the declarations of fp_api_get_state: this branch has INTEGER :: nr, nsa, ntg_last, ntg1_last, #248 has INTEGER :: nr, nsa, ntg_last, nr_alloc, nsa_alloc. The union resolves it; the rest of fp/fp_api.f90 and the three documents both branches edit merge cleanly. The reviewed head 92a0163d conflicts in the same line. If #248 goes in first I will merge develop into this branch and take the union.

python/ on this branch: 730 passed, 35 skipped, 1 xpassed with CI's flags; with the mono image the three test_mono_* files and the pipeline tests, 52 passed. develop has moved to a0b58408 (#242, #243) since your review: this branch merges into it without a conflict, and on develop with #231, #232, #233 and #240 merged together python/ gives 831 passed, 35 skipped, 1 xfailed (#243's strict subTest active). The workflow run needs your approval. #244 and #246 noted; nothing here touches the regress mirror or the struct layout.

HengyuLi-Ozaki-lab and others added 2 commits October 14, 2026 09:24
develop gained k-yoshimi#242, k-yoshimi#243, k-yoshimi#248, k-yoshimi#249, k-yoshimi#251, k-yoshimi#254 and k-yoshimi#255 since this
branch left it. k-yoshimi#248 (fp_get_state bounds its copies by the allocated
extents) touches the same declaration line of fp_api_get_state as this
branch.

One conflict, fp/fp_api.f90: the local variables of fp_api_get_state.
Both sides added names to the same INTEGER statement; the merge keeps
all of them (ntg1_last from this branch, nr_alloc and nsa_alloc from
k-yoshimi#248). The two bodies do not overlap: k-yoshimi#248's copy of the six profiles is
bounded by the allocated extents, and the global scalars of this branch
were already bounded by SIZE(PIT, 1) and SIZE(PIT, 2). In the state k-yoshimi#248
is about (a count changed by fp_set_param after a run) the scalar block
is not executed at all: fp_set_param clears g_prepared, and the scalars
read zero, as this branch's documentation says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the merge of develop, fp_get_state follows two rules in the window
between an fp_set_param and the next run: the six profiles are still the
last run's, within the allocated extents (k-yoshimi#245, k-yoshimi#248), and the seven
global scalars read zero, because they are filled from a prepared state
only. Nothing tested the second in that window; a change of the gate
would have left the suite green.

python/fplib/tests/test_state_scalars_window.py, 8 cases: the state has
the seven; zero before the first run; the volume and the stored energy
after a run, and after a run of zero steps; after a set_param of
NSAMAX + 1, NRMAX + 1 and NRMAX - 1 the seven are zero while timefp and
the profiles are the last run's within the extents k-yoshimi#245 gives (3 x 1,
3 x 1, 2 x 1); the next run fills them again. With the gate changed to
"the arrays are allocated" the three window cases and the last one fail
(measured).

The documents that said "zero until the first run" now also say "and
again after a set_param until the next run that is made, while the
profiles are still those of the last run" (architecture.md, state.md en
and ja, the fplib README, fp_api.h, state.py, the fp_mcp schema text, a
comment in fp_api.f90). "That is made": since k-yoshimi#242 an fp_run can be
refused before it prepares.

CHANGELOG.md gets the entry this pull request did not have.

No code change: the comment in fp_api.f90 is the only Fortran line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HengyuLi-Ozaki-lab

Copy link
Copy Markdown
Collaborator Author

develop (f41fe5bc) is merged in, 0188c139; the conflict was one line.

#248 and this branch both added names to the INTEGER statement of fp_api_get_state; the merge keeps all of them. The two bodies do not overlap: #248 bounds the copy of the six profiles by the allocated extents, and the global scalars here were already bounded by SIZE(PIT, 1) and SIZE(PIT, 2).

The merge puts two rules side by side in the window between an fp_set_param and the next run: the profiles are still the last run's within the allocated extents (#245, #248), and the seven global scalars read zero, because they are filled from a prepared state only. Nothing tested the second in that window, so 7f1b80df adds python/fplib/tests/test_state_scalars_window.py (8 cases: zero before the first run; the volume and the stored energy after a run and after a run of zero steps; after a set_param of NSAMAX + 1, NRMAX + 1 and NRMAX - 1 the seven are zero while timefp and the profiles are the last run's; the next run fills them again). With the gate changed to "the arrays are allocated" four of the eight fail. The documents that said "zero until the first run" now also say "and again after a set_param until the next run that is made". No code change beyond the merge; the CHANGELOG gets the entry this pull request did not have.

On macOS: 838 passed, 35 skipped with the CI flags.

@k-yoshimi

Copy link
Copy Markdown
Owner

Heads-up for the rebase: #246's PR (runtime layout check) is up. Once it lands, fp_state_t has a layout version: FP_STATE_ABI_VERSION is 1 in three places (fp/fp_state.f90 PARAMETER, fp/fp_api.h #define, python/fplib/_ffi.py constant) and libfpapi.so exports fp_state_abi_version() / fp_state_size(). This PR appends seven doubles to the struct, so on rebase it bumps all three to 2, adds a History line in fp_api.h, and fplib/tests/test_ffi.py::TestLibraryLayout plus fp/tests/c_abi/test_negative.c then verify the Fortran type, the header and the ctypes struct agree. No other change needed; the wrapper refuses a stale libfpapi.so with FplibAbiError instead of reading zeros.

HengyuLi-Ozaki-lab and others added 3 commits October 18, 2026 18:39
develop gained k-yoshimi#256, k-yoshimi#257, k-yoshimi#258, k-yoshimi#259, k-yoshimi#261, k-yoshimi#263 and k-yoshimi#264 since the
last merge (f41fe5b).

Two conflicts:
- CHANGELOG.md: this branch's entry and develop's new entries were
  added at the same place under "### Added"; both kept.
- fp/fp_api.f90, the USE lists of the module: this branch reads NTG1,
  TVOLR and the six P*T arrays from fpcomm, k-yoshimi#259 imports
  fp_param_generation; both kept.

And one change that no line conflict shows. k-yoshimi#259 replaced the clear of
g_prepared in fp_api_set_param by a count of registry sets
(fp_param_generation, compared by fp_run with g_prep_generation). The
global scalars of fp_get_state were gated on g_prepared alone, so after
the merge a set no longer zeroed them, against what this branch
documents and tests ("zero ... and again after a set_param until the
next run that is made"). The gate is now "prepared, and no set since
the last prep" (g_prepared .AND. g_prep_generation ==
fp_param_generation()), which also covers sets through tot's "fp:"
dispatcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As asked on the pull request after k-yoshimi#261 (k-yoshimi#246) merged: the seven
doubles this branch appends to fp_state_t make it a new layout, so
FP_STATE_ABI_VERSION goes from 1 to 2 in its three places
(fp/fp_state.f90, fp/fp_api.h, python/fplib/_ffi.py), with a History
line in fp_api.h. A libfpapi.so built before this pull request is 56
bytes shorter, which load_library already refuses on the size since the
merge; one built from the merge alone has the same size at version 1,
and only the version tells it apart. test_ffi.py's
TestLibraryLayout and fp/tests/c_abi/test_negative.c compare the
library, the header and the ctypes struct as before. One test pins the
pairing of version 2 with the seven scalars. In
docs/fp-library/architecture.md the recipe for a layout change names
the three places (it said "in both files", leaving out the Python
constant without which load_library refuses the rebuilt library) and
"currently 2"; the CHANGELOG entry says 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ncluded

Since k-yoshimi#259 a set reaches fp_run's re-prep decision through the
registry's count, and the registry counts fp_set_param_str (KNAMEQ) and
the sets of tot's "fp:" dispatcher as well as fp_set_param. With the
merge's gate ("prepared, and no set since the last prep") the seven
global scalars therefore read zero after any of those until the next
run, not only after fp_set_param. The documents that state the window
say so (architecture.md, fp_api.h, state.md en and ja, the fplib README,
state.py, the fp_mcp schema text, the CHANGELOG entry, the comment at
the gate in fp_api.f90); the test module
no longer says that fp_set_param clears the prepared flag, and a new
case checks the window after set_param_str("KNAMEQ", ...).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HengyuLi-Ozaki-lab

Copy link
Copy Markdown
Collaborator Author

develop (963ae3dc) is merged in, 6118f9a6, and the layout version is bumped as you described.

  • The merge. Conflicts in CHANGELOG.md and the USE lists of fp/fp_api.f90 (both sides kept). One change that no conflict showed: since fix(tr,fp,ti): re-prep after any registry set, including tot's dispatcher (#253) #259 a set no longer clears g_prepared, so the gate of the seven scalars in fp_api_get_state is now g_prepared .AND. g_prep_generation == fp_param_generation(). With g_prepared alone they no longer read zero after a set, and the window tests fail.
  • 1146a444: FP_STATE_ABI_VERSION is 2 in fp/fp_state.f90, fp/fp_api.h (with the History line) and python/fplib/_ffi.py; one test pins version 2 to the seven scalars. The recipe in docs/fp-library/architecture.md said "in both files"; it now names the Python constant as the third place.
  • 4db47e34: with the registry counting every set, the zero window opens after fp_set_param_str and tot's fp: parameters too; the documents say so, with one more case (set_param_str("KNAMEQ", ...)).

macOS, clean build: 980 passed, 38 skipped with the CI flags; the mono-image tests of the CI job with MONO_LIB_PATH set, 41 passed (3 need TOTLIB_PATH as well); the header, libfpapi.so and libtotapi_mono.so all report version 2 / 38488 bytes.

@k-yoshimi k-yoshimi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta review of 6118f9a6 (merge of develop 963ae3dc) + 1146a444 + 4db47e34, in-house + independent Codex. No HIGH, no MED. Approving the delta; the LOWs below are small enough to fold into one push before the merge.

Verified

  • The gate change in the hand resolution is correct: fp_api_run re-preps iff .NOT. g_prepared .OR. g_prep_generation /= fp_param_generation() and records the generation only after a successful fp_prep; fp_api_get_state fills the seven scalars iff g_prepared .AND. g_prep_generation == fp_param_generation(). The two predicates are exact complements, so "scalars non-zero" ⇔ "the next run would not re-prep". Walked all six paths against the code: set after run → zero (profiles kept); set_param_str → same; refused set → no generation bump → unchanged; tot fp: dispatcher (counting fp_param_set) → zero; run again → non-zero; finalize/re-init → closed by g_prepared. A failed fp_param_check returns before the generation, so the window persists.
  • Version 2 is in all three runtime-compared places (fp_state.f90:38, fp_api.h:62, _ffi.py:49); no tot/mono copy needs a bump (test_set_after_run.py imports FpStateC). Size 38488 on both sides, seven trailing doubles at 38432..38480.
  • CHANGELOG resolution kept both entries intact. The window tests are not vacuous: > 0 before the set, all-zero after, > 0 after the next run; the old IF (g_prepared) would fail them.
  • Local: clean build of this branch (macOS, gfortran 15), 273 passed with the CI flags over fplib / totlib / fp_mcp / abi_check (the one failure is Python 3.10's missing ExceptionGroup, pre-existing, CI is 3.11+).

LOW, please fold in

  1. python/mcp-servers/fp_mcp/server.py:203-204 names set_param_str in STATE_SCHEMA while :100 still says "fplib has no set_param_str" (false since fplib.py:163) and this branch exposes no such tool (that is #231). Make the file consistent with this branch: either drop the tool name here or say it is fplib-level until #231.
  2. docs/fp-library/architecture.md:227 "String parameters not wired" is stale for the same reason.
  3. python/fplib/state.py:47-48 and the module docstring of test_state_scalars_window.py:7-9 still say "filled from a prepared state only"; the gate is prepared-and-current-generation. Behaviour sentence is right, mechanism wording is stale.
  4. CHANGELOG.md:25 says "since #259", the code comments and develop's own Changed entry say #253 (the issue). Pick one.
  5. Coverage, optional: no case pins that a refused set leaves the scalars intact, nor that a tot fp: set zeroes them (test_set_after_run.py checks extents only). Both are correct by construction today; a case each would keep them so.

From the approving review of 2026-10-18:
1. fp_mcp/server.py, the state schema: the scalar window is described
   with set_param only. This server has no set_param_str tool on this
   branch (k-yoshimi#231 adds it); the module's comment that fplib has no
   set_param_str is left for k-yoshimi#231, which edits that part of the file.
2. docs/fp-library/architecture.md and the same sentence of the fplib
   README: "String parameters not wired" was stale; fp_set_param_str
   (Fplib.set_param_str) sets KNAMEQ, and the other string keys stay at
   their defaults.
3. python/fplib/state.py and the docstring of
   test_state_scalars_window.py: the scalars are filled only while the
   state is prepared and no parameter was set since that preparation
   (was "from a prepared state only").
4. CHANGELOG.md: k-yoshimi#253, the issue, as in the code comments (was k-yoshimi#259).
5. Two cases: a set the registry refuses leaves the scalars as they were
   (test_state_scalars_window.py); a set of tot's fp: parameters makes
   them read zero, on the mono image (test_set_after_run.py, the file
   CI's mono step runs; it checks the return code of both
   fp_get_state calls, since a failed read also leaves zeros). With the
   gate on g_prepared alone, the mono case fails (measured on a copy, fp
   and the mono image rebuilt).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HengyuLi-Ozaki-lab

Copy link
Copy Markdown
Collaborator Author

The five LOW points are in one commit, 372aeaa1:

  1. The fp_mcp state schema describes the window with set_param only, since this branch has no set_param_str tool. The module comment at server.py:100 ("fplib has no set_param_str") is left for feat(fp_mcp): expose set_param_str tool for KNAMEQ / MODELG=3 equilibrium load #231, which adds the tool and edits that part of the file; this head and feat(fp_mcp): expose set_param_str tool for KNAMEQ / MODELG=3 equilibrium load #231's merge without a conflict. Whichever of the two merges second should say "or set_param_str" in the schema again: the tool opens the same window.
  2. docs/fp-library/architecture.md and the same sentence of the fplib README: "String parameters not wired" now says that fp_set_param_str sets KNAMEQ and the other string keys stay at their defaults.
  3. state.py and the docstring of test_state_scalars_window.py describe the gate as it is: prepared, and no parameter set since that preparation.
  4. CHANGELOG.md says tot: tr: parameter updates after a run bypass tr_api's prepared flag (no re-prep, no #189 pre-run check) #253, as the code comments do.
  5. Both cases: a refused set leaves the scalars as they were; a set of tot's fp: parameters makes them read zero, on the mono image (test_set_after_run.py, TestFpScalarsAfterTotSetOnMonoImage, which checks the return code of both fp_get_state calls, since a failed read also leaves zeros). With the gate on g_prepared alone the mono case fails, along with five of the window cases.

macOS: the whole suite 981 passed, 39 skipped with the CI flags; the mono case passes with MONO_LIB_PATH set. No merge of develop was needed: this head merges cleanly with 4b5463e4.

@k-yoshimi k-yoshimi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta review of 372aeaa1 (the five LOW items) through the full review cycle (Codex + in-house domain/test-oracle lenses + docs/workflow review).

Verdict: approve. Every statement the delta adds was checked against the Fortran: the gate is g_prepared .AND. g_prep_generation == fp_param_generation() (fp/fp_api.f90:370), the scalars are zeroed before every early return of fp_get_state, tot's fp: dispatcher calls the counted fp_param_set wrapper, fp_set_param_str accepts KNAMEQ only, and #253 is what the code comments cite. The new mono-image test is the real discriminator: on a g_prepared-only gate it fails, and a tot dispatch that bypassed the counted wrapper would fail it too. It runs in the CI mono job (python-tests.yml runs test_set_after_run.py with MONO_LIB_PATH), green at 372aeaa1. Locally (no mono image here): 732 passed, 287 skipped, 1 known Python 3.10 failure.

LOW, for the same push or #231, your choice:

  1. fp_mcp/server.py get_state tool docstring (lines ~578-583) lists the returned keys and omits scalars, the block this PR adds.
  2. fp_mcp/README.md §8.4 is the third copy of the "string parameters not wired" sentence the PR corrected in architecture.md and fplib/README.md.
  3. The raw-ctypes mono test hands FpStateC to the image with no layout check; assert lib.fp_state_size() == ctypes.sizeof(FpStateC) in _load_image (which _FpReprepCase would share) closes that.

Optional: test_a_refused_set_leaves_the_scalars observes only through the gate it tests; adding fp.run(1) afterwards and asserting timefp doubles (continuation, no re-prep) would prove the counter did not move independently of fp_get_state. The server.py:100 comment stays as you said, for #231.

Pre-existing docs drift found on the way (symbol count "5" in two READMEs and architecture.md while libfpapi.so exports 8, set_param_str missing from the API reference) is filed as #273; not for this PR.

Merge waits for the maintainers' instruction.

@k-yoshimi
k-yoshimi merged commit ea9806a into k-yoshimi:develop Oct 20, 2026
9 checks passed
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 21, 2026
No line conflict. What the merge changes for this branch without one:
- k-yoshimi#233 put a scalars object into fp_mcp's state schema, zero after a
  set until the next run; set_param_str opens that window too, which
  the schema text did not say yet (next commit).
- k-yoshimi#256 brought eq_mcp's save tool, which this branch's docs still
  called future, after k-yoshimi#226 (fixed in a later commit).
- k-yoshimi#232 and k-yoshimi#257 moved fp_mcp's fd isolation into python/_fd_isolation;
  this branch's server.py keeps it.
- k-yoshimi#261 added the layout queries fp_state_abi_version and fp_state_size
  to libfpapi.so; fplib refuses a library without them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 21, 2026
With develop merged in, this branch has both k-yoshimi#233's scalars object in
the state schema and the set_param_str tool. The schema said the
scalars are zero again after a set_param until the next run; a
successful set_param_str opens the same window (fp_param_set_str moves
the registry's generation counter, fp/fp_param_registry.f90:78, and
fplib's test_state_scalars_window.py checks it for KNAMEQ), so the
description says "after a set_param or set_param_str". Asked on k-yoshimi#231
and in the note on k-yoshimi#233. The module comment above SupportedValue is
this branch's (str is in the union, set through set_param_str), as the
merge kept it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 21, 2026
With develop in this branch, a few sentences no longer hold:
- eq_mcp's save tool is in develop (k-yoshimi#256), so "once k-yoshimi#226 lands" goes
  from the module docstring, the set_param_str tool docstring and
  README section 8.4.
- The get_state tool docstring lists the returned keys and now names
  scalars too (the first LOW of the k-yoshimi#233 approval, left for this PR).
- libfpapi.so exports eight C symbols since k-yoshimi#261: README section 2 now
  names the six entry points and the two layout queries fplib checks.
- The Sphinx FP MCP pages (en, ja) still said nine tools and that the
  string setter is not reachable through MCP; they now list ten tools,
  with a set_param_str row, and point at README section 8.4.
- fp_mcp/__init__.py's docstring said nine tools.
- The comment above SupportedValue says that this server exposes the
  string setter as the set_param_str tool, as suggested on k-yoshimi#231.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 21, 2026
…s the run going

Two points of the k-yoshimi#233 approval, offered for k-yoshimi#233's last push or this
one, and not in k-yoshimi#233 as merged:
- The raw-ctypes mono tests (python/totlib/tests/test_set_after_run.py)
  hand FpStateC and TiStateC to the image without the layout check the
  per-module wrappers do on load (k-yoshimi#246). _check_layout runs that same
  check (_abi_check.check_state_layout: layout version and size, with
  the mono image's rebuild hint) before the first call, in
  _drive_through_tot (the fp and ti re-preparation cases) and in the
  fp scalars case. With the expected version one higher, or the struct
  8 bytes larger, all three fail.
- test_a_refused_set_leaves_the_scalars observed the counter only
  through fp_get_state's gate. It now runs one more step and asserts
  that TIMEFP doubles: the run continues, with no re-preparation. With
  a successful set in place of the refused one, that assertion fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 21, 2026
develop now carries the pieces split out of this branch (k-yoshimi#256 eq_save,
k-yoshimi#257 the MCP fd isolation, k-yoshimi#265 the mcp pin) and k-yoshimi#232, k-yoshimi#233, k-yoshimi#240, k-yoshimi#242,
k-yoshimi#243, k-yoshimi#248, k-yoshimi#249, k-yoshimi#251, k-yoshimi#254, k-yoshimi#255, k-yoshimi#258, k-yoshimi#259, k-yoshimi#261, k-yoshimi#263, k-yoshimi#264. 43
files conflicted; each was resolved as follows.

- Taken from develop (36 files): every file where this branch only
  changed the mcp pin to "mcp>=0.9,<2" (develop has ">=1.2,<2", k-yoshimi#265):
  the seven pyproject.toml, six READMEs, the fp, ti, tot, wr and wrx
  servers, the Sphinx MCP pages, the trlib presentation and its
  builder, the 2026-04-18 MCP plan; and the eq_save pieces now in
  develop in their reviewed form: eq/eq_api.f90, eq/eq_api.h,
  python/eqlib/tests/test_save.py, eq_mcp/tests/test_save_tool.py; and
  tr_mcp/server.py, whose changes here were this branch's own fd
  isolation (develop's _fd_isolation replaces it) and the os._exit(0)
  at the end of main().
- The os._exit(0) that ended main() of tr_mcp and eq_mcp on this branch
  (to skip the Python teardown, cited in k-yoshimi#227 item 5) is gone with
  them: develop's main() returns 0, as approved in k-yoshimi#257. The abort it
  avoided was a double DEALLOCATE, which this branch fixed in
  tr/trcomm.f90 (DEALLOCATE_TRCOMM returns when nothing is allocated);
  that fix stays, merged without a conflict.
- eq/eqfile.f90: develop's (EQSAVE without an argument, EQSAVE_IERR
  doing the work, and its kept EQDSKR error), plus this branch's
  MODELG=25 in EQ_READ's EQDSK branch (k-yoshimi#228-2). eq/eqmenu.f90 merged
  without a conflict but kept this branch's "CALL EQSAVE(IERR)"; it is
  develop's "CALL EQSAVE" again, as EQSAVE takes no argument.
- python/eqlib/_ffi.py, eqlib.py, tests/test_eqlib.py: develop's, plus
  this branch's get_psi_rz (the eq_common_get_psi_rz_ prototype, the
  Eq.get_psi_rz method, its closed-handle assertion).
- eq_mcp/server.py: develop's (the save tool of k-yoshimi#256, the shared fd
  isolation), plus this branch's get_psi_rz handler and tool and its
  --print-tools entry (13 tools). eq_mcp/tests/test_server.py: this
  branch's (13 tools with save and get_psi_rz), which contains
  develop's change.
- python/mcp-servers/eq_mcp/tests/test_import_isolation.py (this branch
  only, no conflict) is removed: it pinned this branch's own isolation
  helper (_install_fd_isolation, _FD_ISOLATION_INSTALLED in eq_mcp and
  tr_mcp), which develop's python/_fd_isolation.py replaces; develop's
  tests/test_fd_isolation.py of each of the seven servers checks the
  same contract (an import leaves fd 1 and sys.stdout alone).
- .github/workflows/python-tests.yml: develop's mcp pin; this branch's
  numpy install (the get_psi_rz tests import numpy) and ../task symlink
  step kept.
- CHANGELOG.md did not conflict. The three k-yoshimi#228 entries this branch held
  back in d8b6e4d are added to [Unreleased] Fixed, after its first
  entry, where no other open pull request inserts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants