Skip to content

Preserve per-frame intrinsics in camera conditioning - #65

Open
zjn20030811 wants to merge 1 commit into
Robbyant:mainfrom
zjn20030811:fix/preserve-per-frame-intrinsics
Open

Preserve per-frame intrinsics in camera conditioning#65
zjn20030811 wants to merge 1 commit into
Robbyant:mainfrom
zjn20030811:fix/preserve-per-frame-intrinsics

Conversation

@zjn20030811

Copy link
Copy Markdown

Summary

  • preserve frame-varying fx, fy, cx, and cy instead of repeating the first calibration across every latent frame
  • resample intrinsics on the exact same target timeline used for pose interpolation in both standard and fast inference
  • keep singleton calibration and action-to-camera trajectories on the existing static-intrinsics path
  • validate short/malformed calibration sequences and retain dtype/device, including bfloat16 inputs

Motivation

The documented camera-control input contains one [fx, fy, cx, cy] row per frame. Both inference paths currently reduce that array to Ks[0] before constructing ray/Plücker conditioning, so zoom or other dynamic calibration is lost while the pose remains time-varying.

The included examples/01 sequence has 561 per-frame rows and roughly a 29-pixel focal-length range. Repeating its first row changes the resulting edge rays by up to about 1.52 degrees. This patch aligns calibration and pose on one shared source/target grid before ray construction.

Compatibility

  • a single calibration row is still repeated
  • action-to-camera trajectories still use the first row because their generated pose timeline does not correspond to the external calibration timeline
  • extra calibration rows are truncated with the pose sequence; a non-singleton short sequence now raises a clear error

Validation

  • python -m pytest tests/test_cam_utils.py -q — 13 passed
  • real example-array smoke checks for standard (41 frames) and fast (39 frames) pose/K/Plücker alignment
  • full Ruff check for the new tests, fatal-rule checks for changed source files, py_compile, and git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant