Repository navigation
release: 4.0.5 — polars 2.0 rolling warm-start fix - #347
Merged
Merged
Conversation
…s-join order) Warm rolling reuse pushes the new p_inflow into nodeBalance_eq as a positional vector built from compute_nodeBalance_dt. The constraint rows are in `over` sorted by its columns (add_cstr determinism wrapper), and the vector silently relied on the lazy cross join nodeBalance x dt coming out left-major. polars 2.0 no longer guarantees that: with a small left side it emits right-major order ((heat,t1),(west,t1),...), so every other (n, t) inflow landed on the wrong node. In test_delayed_input_share_rolling_csv_chain roll 1/2 (warm-reused) west demand became 0 in every other hour and the unit stopped there (objective 1.007e9 vs the correct 1.577e9) - CI-only, polars 2.0.0. - _warm: sort new_over by its columns before building the vector and keep the left join's order explicitly (maintain_order="left"). - _pdt_join: all six lazy cross-join helpers go through _cross(...) with maintain_order="left_right", restoring left-major (sorted) output on polars 2 - also keeps the determinism wrapper's already-sorted fast path for the big (entity, d, t) grids. - test_warm_rhs_alignment: both fail on the old code under polars 2. Works on polars 1.40 and 2.0 (maintain_order exists in both). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cai1ke9MzbEMpY1jfScB6q
End-to-end guard for the polars 2 warm-start misalignment: tests.json + override alternative, coal_chp serving west (1000 MW) and heat (0 MW), rolled 24 h / 12 h. Asserts at least one roll reuses the warm problem and every roll's warm objective equals the cold one. On the unfixed code under polars 2.0.0 roll_1 came out at 1.387e9 vs cold 2.774e9 (west demand moved onto heat every other hour); none of the existing warm/cold equivalence fixtures has two nodes with different demand in a warm-reused rolling cascade, so none caught it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cai1ke9MzbEMpY1jfScB6q
version 4.0.4 -> 4.0.5 + CHANGELOG. Ships the warm-start RHS alignment fix: with polars 2.0 (pulled by fresh installs; polars>=1.40 has no upper bound) rolling runs that reuse the built problem between same-length rolls pushed inflow values by position in an order polars 2 no longer guarantees, silently misplacing demand between nodes/time steps. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cai1ke9MzbEMpY1jfScB6q
DbEditorManager (used by main_window to open input .sqlite sources and
the comparison results.sqlite) looked the editor up with
shutil.which("spine-db-editor"). flextool-gui started from a venv that
is not activated has no venv bin on PATH, so it reported the editor
missing while the updater (importability check) said Spine Toolbox was
installed. 26f1588 fixed only platform_utils.open_spine_db_editor.
- platform_utils: one shared helper set - spine_toolbox_available()
(importlib find_spec, no PATH) and spine_db_editor_argv(urls)
([sys.executable, "-m", "spinetoolbox.spine_db_editor.main", *urls]).
- DbEditorManager.open_database uses them (one URL or several as tabs);
process tracking and launch-log capture unchanged; clear warning naming
the interpreter when spinetoolbox is not importable.
- open_spine_db_editor and the CLI's _build_launch_argv go through the
same helper so the launchers cannot drift apart again.
- main_window messages: "Spine Toolbox is not installed in the Python
environment running FlexTool" instead of "'spine-db-editor' command
was not found".
- tests/gui/test_db_editor_launch.py: Popen mocked, stub spinetoolbox
package, PATH without any spine-db-editor.
- CHANGELOG 4.0.5 entry.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cai1ke9MzbEMpY1jfScB6q
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug (affects released 4.0.4)
pyproject.tomlrequirespolars>=1.40with no upper bound, so fresh installs get polars 2.0.0. polars 2.0 no longer keeps left-major order for a lazy cross join (nor left order for a lazy left join) unlessmaintain_orderis given.In warm-start rolling, a roll with the same LP shape as the previous one reuses the built
WarmProblemand pushes the newp_inflowintonodeBalance_eqby position. That vector came fromcompute_nodeBalance_dt(lazynodeBalance × dtcross join), assumed to be in the constraint's sorted row order. Under polars 2.0 it comes out right-major, so with ≥2 balance nodes every other(n, t)inflow lands on the wrong node — silently (e.g. one node's demand zero every other hour).Fix (cherry-pick of 3fb9988, applied cleanly)
_warm.py: sort the pushednew_overby its columns (the cstr's own row order, set by the add_cstr determinism wrapper) andmaintain_order="left"on the value left-join._pdt_join.py: the six lazy cross-join helpers go through_cross()withmaintain_order="left_right".test_warm_rhs_alignment.py(unit) + newtest_warm_rolling_multinode.py(e2e: two nodes, rolling 24h/12h, warm == cold per roll, asserts warm reuse actually happens). The existing warm/cold equivalence fixtures did not catch this (no two-node, different-demand, warm-reused rolling cascade).Verification
pip install -e .→ polars 2.0.0, polar-high 3.8.0 from PyPI, highspy 1.14.0): fullpytest tests/(FLEXTOOL_AUTOSCALE_STRICT=1) — 3622 passed, 235 skipped, 1 xfailed, 0 failed.tests/— 3632 passed, 225 skipped, 1 xfailed, 0 failed.maintain_order— only one (benign, output row-order only) assertion tripped; no other positional dependence in the warm/LP path. LP col/row ids are order-independent (determinism wrapper sorts before id assignment).Also: GUI Spine DB editor launch (832126c)
DbEditorManager(main window: open input .sqlite / comparison results.sqlite) found the editor viashutil.which("spine-db-editor"), soflextool-guistarted from a non-activated venv reported it missing although Spine Toolbox was installed. It now launches[sys.executable, "-m", "spinetoolbox.spine_db_editor.main", *urls]after an importability check, via one helper inplatform_utilsshared withopen_spine_db_editorand the CLI launcher. Test:tests/gui/test_db_editor_launch.py(Popen mocked, stubspinetoolbox, PATH without the editor) — passes with polars 2.0 and 1.40 envs;tests/gui+tests/test_open_results_db.pygreen in both.Version 4.0.4 → 4.0.5 + CHANGELOG.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Cai1ke9MzbEMpY1jfScB6q