Skip to content

Fix indexing errors in particle smoothers - #282

Merged
SamDuffield merged 2 commits into
state-space-models:mainfrom
DanWaxman:dw-backward-sampler-fix
Oct 2, 2026
Merged

SamDuffield merged 2 commits into
state-space-models:mainfrom
DanWaxman:dw-backward-sampler-fix

Conversation

@DanWaxman

Copy link
Copy Markdown
Collaborator

Resolves #281. This fixes an indexing bug in the particle smoothers, which results in the incorrect smoothing distribution. Concretely, state_2.model_inputs was previously used as part of the log_potential:

new_particles_1, ancestors_1 = backward_sampling_fn(
state_1.key,
x0_all=state_1.particles,
x1_all=state_2.particles,
log_weight_x0_all=state_1.log_weights,
log_density=lambda s1, s2: log_potential(s1, s2, state_2.model_inputs),
x1_ancestor_indices=state_2.ancestor_indices,
)

But the model_inputs convention for particle smoothers differ from the rest of cuthbert, and state_1.model_inputs should be used accordingly:

def smoother_prepare(
filter_state: ParticleFilterState,
model_inputs: ArrayTreeLike,
key: KeyArray | None = None,
) -> ParticleSmootherState:
"""Prepare a state for a particle smoother step.
Note that the model_inputs here are different to filter_state.model_inputs.
The model_inputs required here are for the transition from t to t+1.
filter_state.model_inputs represents the transition from t-1 to t.

The proper state_1.model_inputs is now properly used.

This issue wasn't found by the tests because the model definition was wrong in tests: in particular, backward sampling smoothers require the joint log potential, but only the observation log potential was given. We now use the proper model definition, which would fail before but passes now.

Finally, there was a key reshape that caused issues with typed keys. We now split with the correct shape directly, which makes this compatible with both JAX key types.

Resolves state-space-models#281. This fixes an indexing bug in the particle smoothers, which results in the incorrect smoothing distribution. Concretely, `state_2.model_inputs` was previously used as part of the `log_potential`:

https://github.com/state-space-models/cuthbert/blob/ec53db22e17b6ab29975e41af8a0758a2a6ff665/cuthbert/smc/backward_sampler.py#L181-L188

But the `model_inputs` convention for particle smoothers differ from the rest of `cuthbert`, and `state_1.model_inputs` should be used accordingly:

https://github.com/state-space-models/cuthbert/blob/ec53db22e17b6ab29975e41af8a0758a2a6ff665/cuthbert/smc/backward_sampler.py#L128-L137

The proper `state_2.model_inputs` is now properly used.

This issue wasn't found by the tests because the model definition was wrong in tests: in particular, backward sampling smoothers require the joint log potential, but only the observation log potential was given. We now use the proper model definition, which would fail before but passes now.

Finally, there was a key reshape that caused issues with typed keys. We now split with the correct shape directly, which makes this compatible with both JAX key types.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

An unused variable fails Ruff checks, and the increased particle count substantially expands quadratic test work.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Fixes particle-smoother input indexing and typed JAX key compatibility.

Changes:

  • Uses the correct transition inputs during backward sampling.
  • Corrects the smoothing test model.
  • Splits MCMC keys directly into the required shape.
File Description
cuthbert/​smc/​backward_sampler.py Corrects smoother input indexing.
cuthbertlib/​smc/​smoothing/​mcmc.py Supports typed and legacy keys.
tests/​cuthbert/​smc/​test_backward_sampler.py Updates the smoothing regression test.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/cuthbert/smc/test_backward_sampler.py
Comment thread tests/cuthbert/smc/test_backward_sampler.py
Tightens the MSE threshold, and tests covs.

@SamDuffield SamDuffield left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch and thanks a lot for the fix!!

@SamDuffield
SamDuffield merged commit 18d3dc9 into state-space-models:main Oct 2, 2026
2 checks passed
@DanWaxman
DanWaxman deleted the dw-backward-sampler-fix branch October 2, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Indexing Bug in Particle Smoothers

3 participants