Skip to content

Serve the raw sensor frame intact, and add /api/camera/rawfull - #559

Closed
mrosseel wants to merge 1 commit into
brickbots:mainfrom
mrosseel:api/camera-raw-frames
Closed

Serve the raw sensor frame intact, and add /api/camera/rawfull#559
mrosseel wants to merge 1 commit into
brickbots:mainfrom
mrosseel:api/camera-raw-frames

Conversation

@mrosseel

Copy link
Copy Markdown
Collaborator

Two changes so the frames the pipeline actually works on can be retrieved from a running device.

/api/camera/raw was returning noise

It rendered the uint16 sensor frame with:

img = Image.fromarray(arr, mode="L").convert("RGB")

mode="L" reinterprets the 16-bit buffer as 8-bit, so you get interleaved high/low bytes rather than an image. The failure is nasty because the output still looks like a plausible frame — I pulled 40-odd captures off a device with it while debugging a solve failure and only noticed when the histogram made no sense: median 1.0, p90 80, over 1% of pixels at 255. Bimodal byte noise, not sky.

16-bit frames now render as I;16 PNGs, preserving every ADU for offline analysis. A test pins the round trip on real 12-bit values.

/api/camera/rawfull (new)

The whole sensor, including the margins the square crop discards. Naming follows the exposure sweep's TIFFs — raw is the crop, rawfull is everything.

The full frame is ~4 MB, so publishing it on every capture would put that across the state manager continuously. It's served on demand instead: the endpoint sets a request flag, the camera fulfils it on the next capture and clears it, so one request yields exactly one frame. A test pins that one-shot behaviour.

Why

Debugging a plate-solve failure in the field currently offers no way to see what the sensor actually delivered. The cropped frame was corrupt and the full frame was unreachable, so the only diagnosis available was indirect — inferring from radiometer telemetry that light was being attenuated somewhere ahead of the sensor. Being able to fetch both frames turns that into looking at the picture.

Testing

ruff and mypy clean, 1077 unit+smoke tests pass, including the two new ones.

/api/camera/raw rendered a uint16 sensor frame with
Image.fromarray(arr, mode="L"), which reinterprets the 16-bit buffer as
8-bit. The result is interleaved-byte noise that still looks enough like an
image to be believed -- I spent a night's captures on it before checking the
histogram (median 1, p90 80, 1% at 255) and realising the frames were junk.

16-bit frames now render as mode "I;16" PNGs, preserving every ADU, and a
test pins the round trip.

Adds /api/camera/rawfull for the whole sensor including the margins the crop
discards. Naming follows the exposure sweep's TIFFs: raw is the crop,
rawfull is everything. The full frame is ~4 MB, so it is published on demand
rather than on every capture -- the endpoint asks, the camera fulfils the
request once and clears it.

Between them these make the frames the pipeline actually works on
retrievable from a running device, which is what debugging a solve failure
in the field needs and what nothing currently offers.
@mrosseel

Copy link
Copy Markdown
Collaborator Author

Folded into #554 — the raw/rawfull API is part of the same work, no reason for it to be a separate PR.

@mrosseel mrosseel closed this Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant