Skip to content

fix(tr,fp,ti): re-prep after any registry set, including tot's dispatcher (#253) - #259

Merged
k-yoshimi merged 2 commits into
developfrom
fix/registry-generation-253
Oct 17, 2026
Merged

k-yoshimi merged 2 commits into
developfrom
fix/registry-generation-253

Conversation

@k-yoshimi

@k-yoshimi k-yoshimi commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Fixes #253 (develop-targeted, so close by hand after merge).

Why

tr_set_param / fp_set_param / ti_set_param cleared the module's private g_prepared flag themselves. tot's tr: / fp: / ti: dispatcher (tot/tot_param_registry.f90) calls tr_param_set & co. directly, so a set through Tot (or tot_mcp) after a run left the flag standing: the next tot_run continued the old state with tr_prep skipped, the #189 pre-run species check never fired, and Tot.set_param("tr:NSMAX", 3) + run could still produce NaN. TotPipeline was not affected (it goes through the per-module wrappers), so the two tot front ends disagreed silently.

What

The registry counts, the API compares. Each of tr_param_registry, fp_param_registry, ti_param_registry gets a private g_generation that every successful set advances (the old bodies become *_param_set_impl; the public names are two-line wrappers, so an early RETURN inside the SELECT CASE cannot skip the count and a refused set leaves it alone), and a read-only accessor <mod>_param_generation(). tr_api / fp_api / ti_api keep g_prepared, add g_prep_generation, and re-prep when .NOT. g_prepared .OR. g_prep_generation /= <mod>_param_generation(); the explicit g_prepared = .FALSE. in the setters is gone, so there is one mechanism. Whoever reaches the registry (the C ABI setter, tot's dispatcher, a future caller) invalidates the prep.

Scope: *_param_registry.f90 (L-3 registry) and *_api.f90 (L-2 C ABI) only. No trprep / trloop / physics change, so this is outside the lead-sign-off rule for Fortran body changes.

Behaviour change, Tot and tot_mcp only: a set after a run now restarts the module (NT, T back to 0), which is what trlib / fplib / tilib / TotPipeline have always done. No test or baseline relied on the old continue semantics (checked: no Tot test sets a tr: parameter after a run and runs again; totregress.f90 reads a namelist). A "live" coupling input that should not restart the module (EXTERNAL_DRIVEN_I is the candidate) would need a live/structural classification in the registry; that is a follow-up, not this PR.

eq has no prepared flag on develop, so no counter there; #256's g_has_equilibrium is the natural consumer and I will suggest the same shape on that PR.

Tests (python/totlib/tests/test_set_after_run.py, 5 cases)

  • Tot: run(2) → nt == 2, set_param("tr:DT"), run(1) → nt == 1; Trlib with the same sequence agrees.
  • Tot: run(1), set_param("tr:NSMAX", 3) (species 3 has PN = PNS = 0), run(1) raises: the tr_tst2 fixture: NSMAX>2 mutation produces NaN scalars (RNU/RTU species 3+ unpopulated) #189 check is reached. Fails on develop.
  • A set the registry refused does not restart (nt == 3).
  • fp and ti on the mono image (MONO_LIB_PATH, CI's mono routing step; skipped elsewhere) with raw ctypes: tot_init, <mod>_run, tot_set_param("<mod>:NRMAX", 20), <mod>_run. fp: fp_get_state().nrmax == 20 (it reports the allocated extent, fix(fp): bound fp_get_state copies by the allocated extents (#245) #248, so this holds only if fp_prep reallocated). ti: ti_get_state reports NRMAX straight from the namelist, so the observable is NT: ti_run(2), set, ti_run(1) ends at nt == 1 only after a re-prep (3 when the old state continues). The ti case uses tilib's TiDataCwdMixin for the ADPOST data cwd. Not on the default libtotapi.so: it links the sibling lib*api.so, each with its own PIC copy of pl / eq / bpsd, and on Linux ELF interposition binds the duplicates to the first-loaded copy, so fp_run reached through it mixes copies and crashed on CI (SIGSEGV / SIGBUS; macOS's two-level namespaces hid it locally). Running fp / ti alongside tot in one process is what the mono image is for.

Verification

  • Local (macOS gfortran): tr/fp/ti .so, libtotapi.so, libtotapi_mono.so rebuilt; the new file 3 passed + 2 mono-skipped without MONO_LIB_PATH, 5 passed with it; pytest python/totlib/tests python/trlib/tests python/fplib/tests python/tilib/tests python/mcp-servers/tot_mcp python/mcp-servers/tr_mcp --forked --timeout=120 --timeout-method=signal → see the first comment; make -C tot tot_api_check_so → see the first comment.
  • Against a develop build (fresh clone of develop @ f41fe5b, same test file): the two Tot restart/refusal cases fail (nt 3 instead of 1, no refusal) and, driven the same way against develop's libraries, fp nrmax stayed 1 and ti nt ended at 3; the refused-set case passes; with the fix all pass.
  • Reviews: in-house + Codex (first comment).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TA2KahTs3qLMZL6yjSsWwT

…cher (#253)

tr_set_param / fp_set_param / ti_set_param cleared the module's private
g_prepared flag themselves; tot's "tr:" / "fp:" / "ti:" dispatcher writes
to the registries directly and left the flag standing, so a set through
Tot after a run continued the old state with tr_prep skipped and the #189
pre-run species check never fired.

The registry now counts every successful set (g_generation behind
<mod>_param_generation(); the old bodies are *_param_set_impl, the public
names two-line wrappers so a refused set never counts) and tr_run /
fp_run / ti_run re-prep when the count moved since their last prep. The
explicit clears in the setters are gone: one mechanism, every caller.

Behaviour change for Tot / tot_mcp only: a set after a run restarts the
module, as trlib / fplib / tilib / TotPipeline always did. Tests: Tot vs
Trlib restart, NSMAX=3 after a run is refused, a refused set does not
restart, fp and ti re-prep after tot_set_param on the default and the
mono image (file added to CI's mono routing step).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TA2KahTs3qLMZL6yjSsWwT
@k-yoshimi

Copy link
Copy Markdown
Owner Author

Pre-push reviews (in-house + Codex, both read-only on 52ad20e8 → re-checked at 2eb8cd11; the final 73d8e0a differs only by the two LOW fixes below and the README paragraph).

Verified by the reviewers against the code: the *_param_set → *_param_set_impl rename is complete for every public entry point (tr ×2, fp ×2, ti ×1), no caller references the _impl names, tot's dispatcher USEs only the public wrappers (so it is counted now), the C-ABI drivers call only the C symbols; g_generation is SAVE'd module state that neither init nor finalize reset, which is fine because both set g_prepared = .FALSE. and the run condition ORs the two; a failed prep (and tr's pre-prep refusals: NSM bound, #189 check) returns before the snapshot, so the next run retries; the only uses of g_prepared in the three API modules are declaration / init / run / finalize; per-module wrapper behaviour is unchanged (successful set → count moves → re-prep; refused set → no count → no clear, as before); dlsym on libtotapi.so reaches fp_run / ti_run in its DT_NEEDED lib*api.so (glibc searches the dependency tree; RTLD_LOCAL only hides them from other objects), i.e. the same copies dispatch_fp / dispatch_ti wrote to; fp's observable is the allocated extent (#248), ti's is NT (tiprep.f90:344 resets it); TotlibInvalidParamError / TotlibCalculationFailedError both subclass TotlibError; no existing test or baseline relies on tot's old continue semantics.

Findings, all applied:

  • LOW (in-house): the skip guards were decorator-time skipUnless on mutable preconditions (the pattern Bugbot flagged twice on test: FIXTURES_DIR fallback for sweep + property_fanout (#192) #221); moved into setUp. MONO_LIB_PATH is made absolute before TiDataCwdMixin chdirs.
  • LOW (Codex): README said fp: / ti: restart "on the next Tot.run", but tot_run drives tr_run only; reworded (next fp_run / ti_run).
  • HIGH (Codex), not fixable here, documented: on the mono image several registries write the same plcomm variables (ti_param_registry USEs plcomm for RR/RA/RKAP/RDLT/BB/RIP, so ti:RR and eq:RR are one RR), and a set through one module's registry moves that module's count only. This is the mono image's pre-existing shared-storage property, present on develop exactly the same way (the old per-setter clear was per module too); a plcomm-owned counter would need a pl Fortran body change. Tracked in the follow-up issue linked below, noted in python/totlib/README.md.

Local: pytest python/totlib/tests python/trlib/tests python/fplib/tests python/tilib/tests python/mcp-servers/tot_mcp python/mcp-servers/tr_mcp --forked --timeout=120 --timeout-method=signal → 470 passed, 26 skipped, 1 failed (test_close_raises_exception_group_when_multiple_modules_fail, the known macOS / py3.10 ExceptionGroup case, unrelated). make -C tot tot_api_check_so → OK. New file: 5 passed + 2 mono-skipped on the default image, 7 passed with MONO_LIB_PATH; against a fresh develop build the two Tot cases and the fp / ti cases fail as described in the body.

)

On Linux CI the fp case through the default libtotapi.so crashed
(SIGSEGV / SIGBUS): that image links the sibling lib*api.so, each with
its own PIC copy of pl / eq / bpsd, and ELF interposition binds the
duplicate symbols to the first-loaded copy, so fp_run reached through it
mixes copies. macOS's two-level namespaces hid it locally. The mono image
is the supported way to run fp / ti alongside tot in one process, and its
cases passed on Linux; the two default-image classes are removed and the
reason is in the module docstring.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TA2KahTs3qLMZL6yjSsWwT
@k-yoshimi

Copy link
Copy Markdown
Owner Author

CI on 73d8e0a failed in one test only: test_fp_reprepares_after_tot_set on the default libtotapi.so crashed on Linux (SIGSEGV / SIGBUS), while the same fp / ti cases on the mono image and the three Tot cases passed. Cause: the default image carries five PIC copies of pl / eq / bpsd (one per sibling lib*api.so), and on ELF the linker picks one definer per symbol across the copies, so the state tot_init prepared and the state fp_run touches need not be the same copy; macOS hid it. No product path calls fp_run / ti_run through libtotapi.so after tot_init, so fc74f00 runs those two cases on the mono image only (CI's mono step) and says why in the docstring; the hazard itself is recorded on #260. Both reviewers re-checked the delta (clean / wording applied).

@k-yoshimi
k-yoshimi merged commit 1865115 into develop Oct 17, 2026
3 checks passed
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 18, 2026
develop gained k-yoshimi#256, k-yoshimi#257, k-yoshimi#258, k-yoshimi#259, k-yoshimi#261, k-yoshimi#263 and k-yoshimi#264 since the
last merge (f41fe5b).

Two conflicts:
- CHANGELOG.md: this branch's entry and develop's new entries were
  added at the same place under "### Added"; both kept.
- fp/fp_api.f90, the USE lists of the module: this branch reads NTG1,
  TVOLR and the six P*T arrays from fpcomm, k-yoshimi#259 imports
  fp_param_generation; both kept.

And one change that no line conflict shows. k-yoshimi#259 replaced the clear of
g_prepared in fp_api_set_param by a count of registry sets
(fp_param_generation, compared by fp_run with g_prep_generation). The
global scalars of fp_get_state were gated on g_prepared alone, so after
the merge a set no longer zeroed them, against what this branch
documents and tests ("zero ... and again after a set_param until the
next run that is made"). The gate is now "prepared, and no set since
the last prep" (g_prepared .AND. g_prep_generation ==
fp_param_generation()), which also covers sets through tot's "fp:"
dispatcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 18, 2026
…ncluded

Since k-yoshimi#259 a set reaches fp_run's re-prep decision through the
registry's count, and the registry counts fp_set_param_str (KNAMEQ) and
the sets of tot's "fp:" dispatcher as well as fp_set_param. With the
merge's gate ("prepared, and no set since the last prep") the seven
global scalars therefore read zero after any of those until the next
run, not only after fp_set_param. The documents that state the window
say so (architecture.md, fp_api.h, state.md en and ja, the fplib README,
state.py, the fp_mcp schema text, the CHANGELOG entry, the comment at
the gate in fp_api.f90); the test module
no longer says that fp_set_param clears the prepared flag, and a new
case checks the window after set_param_str("KNAMEQ", ...).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 18, 2026
develop gained k-yoshimi#242, k-yoshimi#243, k-yoshimi#248, k-yoshimi#249, k-yoshimi#251, k-yoshimi#254, k-yoshimi#255, k-yoshimi#256, k-yoshimi#257,
k-yoshimi#258, k-yoshimi#259, k-yoshimi#261, k-yoshimi#263 and k-yoshimi#264 since this branch left it.

One conflict, in python/eqlib/_ffi.py, python/fplib/_ffi.py and
python/trlib/_ffi.py alike: the end of load_library. k-yoshimi#261 (k-yoshimi#246) checks
the state layout of the library just loaded and raises
<Mod>libAbiError on a mismatch; this branch closes the directory it
registered with os.add_dll_directory when the load fails. The check is
now inside the block that cleans up, so a library refused for its
layout does not leave its directory in the process-wide DLL search set
either (or an entry in _DLL_DIR_HANDLES). Order as on develop: the
layout check, then the prototypes.

With the check on the load path, the cases of test_platform_libname.py
that fake the loaded library (ctypes.CDLL patched) were refused by it:
24 of 39 failed. Each of the three files now has an autouse fixture that
patches check_state_layout out for them (the check has its own cases in
test_ffi.py, TestLibraryLayout), and one new case per module puts a
check back that raises AbiMismatchError (with the prototypes passing,
so that only the check can refuse the load) and asserts
<Mod>libAbiError, the directory handle closed and nothing registered;
with the check moved after the cleanup block, that case fails (measured
on eq).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 20, 2026
From the approving review of 2026-10-18:
1. fp_mcp/server.py, the state schema: the scalar window is described
   with set_param only. This server has no set_param_str tool on this
   branch (k-yoshimi#231 adds it); the module's comment that fplib has no
   set_param_str is left for k-yoshimi#231, which edits that part of the file.
2. docs/fp-library/architecture.md and the same sentence of the fplib
   README: "String parameters not wired" was stale; fp_set_param_str
   (Fplib.set_param_str) sets KNAMEQ, and the other string keys stay at
   their defaults.
3. python/fplib/state.py and the docstring of
   test_state_scalars_window.py: the scalars are filled only while the
   state is prepared and no parameter was set since that preparation
   (was "from a prepared state only").
4. CHANGELOG.md: k-yoshimi#253, the issue, as in the code comments (was k-yoshimi#259).
5. Two cases: a set the registry refuses leaves the scalars as they were
   (test_state_scalars_window.py); a set of tot's fp: parameters makes
   them read zero, on the mono image (test_set_after_run.py, the file
   CI's mono step runs; it checks the return code of both
   fp_get_state calls, since a failed read also leaves zeros). With the
   gate on g_prepared alone, the mono case fails (measured on a copy, fp
   and the mono image rebuilt).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengyuLi-Ozaki-lab added a commit to HengyuLi-Ozaki-lab/task that referenced this pull request Oct 21, 2026
develop now carries the pieces split out of this branch (k-yoshimi#256 eq_save,
k-yoshimi#257 the MCP fd isolation, k-yoshimi#265 the mcp pin) and k-yoshimi#232, k-yoshimi#233, k-yoshimi#240, k-yoshimi#242,
k-yoshimi#243, k-yoshimi#248, k-yoshimi#249, k-yoshimi#251, k-yoshimi#254, k-yoshimi#255, k-yoshimi#258, k-yoshimi#259, k-yoshimi#261, k-yoshimi#263, k-yoshimi#264. 43
files conflicted; each was resolved as follows.

- Taken from develop (36 files): every file where this branch only
  changed the mcp pin to "mcp>=0.9,<2" (develop has ">=1.2,<2", k-yoshimi#265):
  the seven pyproject.toml, six READMEs, the fp, ti, tot, wr and wrx
  servers, the Sphinx MCP pages, the trlib presentation and its
  builder, the 2026-04-18 MCP plan; and the eq_save pieces now in
  develop in their reviewed form: eq/eq_api.f90, eq/eq_api.h,
  python/eqlib/tests/test_save.py, eq_mcp/tests/test_save_tool.py; and
  tr_mcp/server.py, whose changes here were this branch's own fd
  isolation (develop's _fd_isolation replaces it) and the os._exit(0)
  at the end of main().
- The os._exit(0) that ended main() of tr_mcp and eq_mcp on this branch
  (to skip the Python teardown, cited in k-yoshimi#227 item 5) is gone with
  them: develop's main() returns 0, as approved in k-yoshimi#257. The abort it
  avoided was a double DEALLOCATE, which this branch fixed in
  tr/trcomm.f90 (DEALLOCATE_TRCOMM returns when nothing is allocated);
  that fix stays, merged without a conflict.
- eq/eqfile.f90: develop's (EQSAVE without an argument, EQSAVE_IERR
  doing the work, and its kept EQDSKR error), plus this branch's
  MODELG=25 in EQ_READ's EQDSK branch (k-yoshimi#228-2). eq/eqmenu.f90 merged
  without a conflict but kept this branch's "CALL EQSAVE(IERR)"; it is
  develop's "CALL EQSAVE" again, as EQSAVE takes no argument.
- python/eqlib/_ffi.py, eqlib.py, tests/test_eqlib.py: develop's, plus
  this branch's get_psi_rz (the eq_common_get_psi_rz_ prototype, the
  Eq.get_psi_rz method, its closed-handle assertion).
- eq_mcp/server.py: develop's (the save tool of k-yoshimi#256, the shared fd
  isolation), plus this branch's get_psi_rz handler and tool and its
  --print-tools entry (13 tools). eq_mcp/tests/test_server.py: this
  branch's (13 tools with save and get_psi_rz), which contains
  develop's change.
- python/mcp-servers/eq_mcp/tests/test_import_isolation.py (this branch
  only, no conflict) is removed: it pinned this branch's own isolation
  helper (_install_fd_isolation, _FD_ISOLATION_INSTALLED in eq_mcp and
  tr_mcp), which develop's python/_fd_isolation.py replaces; develop's
  tests/test_fd_isolation.py of each of the seven servers checks the
  same contract (an import leaves fd 1 and sys.stdout alone).
- .github/workflows/python-tests.yml: develop's mcp pin; this branch's
  numpy install (the get_psi_rz tests import numpy) and ../task symlink
  step kept.
- CHANGELOG.md did not conflict. The three k-yoshimi#228 entries this branch held
  back in d8b6e4d are added to [Unreleased] Fixed, after its first
  entry, where no other open pull request inserts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant