fog demo v2: field of view, moving worlds, squads, and real city geometry - #25
Merged
Conversation
…ette
The demo showed WHAT the agent believed but not WHY it believed it. A viewer
could not tell whether the agent had looked somewhere and found nothing or
simply never looked, and real geometry the agent had not discovered was
invisible to the audience too -- so 'the map is wrong' had to be narrated
rather than seen.
* belief.py exposes the swept field of view. sense() already computed the
free/hit masks for its ray walk; last_visible and ever_seen fall out of
them at no extra cost.
* fog_record captures both on every SENSE tick, not only when belief
changes: the sensor sweeps with the vehicle even on ticks where it learns
nothing, and an overlay that only moved on discoveries looks broken.
1152 bytes per 96x96 snapshot before compression.
* fog_trace reads them back; traces recorded before this simply report
has_fov() False and render without fog.
* fog_scene paints the three tiers. The fog is INVERTED from the obvious
construction: the ground plate IS the darkness and knowledge is painted
onto it, because fog drawn over unseen cells can only cover the world
bounds and leaves the rest of the 16:9 plate lit -- which reads as 'the
map ends here' rather than 'this is unknown'. One fewer mesh, and the
whole frame is fogged.
* SILHOUETTE: truth & ~believed, drawn as a boundary outline so the viewer
sees real geometry the agent has not found. outline_segments() emits an
edge only where a set cell meets an unset one, so a solid block yields its
perimeter rather than a grid of every cell, and add_segments() builds
disjoint line pairs because a boundary is generally several loops (a
polyline would connect them across the scene).
* The sensor's range is drawn as a ring that rides with the vehicle.
Drawing that ring found a real bug in the sensor. The DDA normalises each ray
by its dominant component, so a step is one cell along that axis and a
diagonal ray travelled sqrt(2) farther than an axis-aligned one: the visible
set was a SQUARE and the lit region visibly overflowed the circle. range_m now
means the same thing in every direction. The honest picture is what exposed
the dishonest model.
tests: 11 new (isotropy, occlusion of the FOV, memory vs current visibility,
outline is a perimeter not a grid, disjoint shapes, silhouette is exactly
truth-minus-belief). 77 pass.
Three new scenes, and the machinery they needed. MOVERS are part of TRUTH, not of belief. A unit_at Event stamps the belief layer directly -- fine for 'something was reported here', but it cheats: the agent is simply told. A Mover physically exists, occludes, and has to be SEEN before the agent knows about it; until then the route runs straight at it. That is what makes a second vehicle interesting rather than decorative. truth_now is rebuilt each tick (static world + every mover footprint) and is what the sensor reads and the collision check scores against. A mover the sensor just saw is demoted into the DECAYING layer and cleared from the static map. Without that, a moving vehicle smears a permanent wall along its path: every sweep bakes its current cells into log-odds and nothing ever removes them. MOVING GOAL retargets the active waypoint every tick and forces a replan on the sense cadence. The navigator's track_goal retargets in place, so the route follows the target rather than restarting. New scenes: city (a 3x3 block grid, map stale in two places -- a wrong TURN rather than a wrong metre), traffic (correct map, two vehicles crossing it), and pursuit (a target that runs). All three reach the goal with ZERO truth penetration: city 410 ticks / 34 map updates, traffic 805 ticks / 95 updates while dodging both vehicles, pursuit 632 ticks closing on a goal that fled. AND A BUG THE NEW SCENES EXPOSED: build() drew one red mesh per story.truth_rects unconditionally, so every real building was painted as KNOWN the moment the scene loaded -- ground truth on screen from frame one, which made the fog decorative. The three original stories carry no truth_rects, so this stayed invisible until a scene with real geometry existed. Truth now reaches the frame only through belief: known (red), wrongly believed (amber), undiscovered (outline). tests: 79 pass. Two new regressions -- build() must not draw ground truth, and wall/ghost/silhouette must partition what is drawn (disjoint, and together covering everything either real or believed).
…reets
Two bugs, both cosmetic-looking and both wrong in a way that misrepresents the
simulation.
1. INVERTED COLOURS on any geometry that arrives via an event. apply() split
belief into wall/ghost using story.truth_grid() -- the truth as DECLARED at
t=0. The blocker story's truth_rects is empty and its wall is raised by an
add_rect event at step 40, so occ & truth was always empty: every
discovered cell of a real wall was painted GHOST amber ('believed but
absent') and the red 'wall' node never appeared at all. The shipped
deliverable video has this.
Ground truth CHANGES -- events raise and demolish walls, movers drive
around -- so 'is this believed cell actually real?' is a question about the
world at that moment. The trace now carries a truth snapshot per sense tick
and the renderer classifies against truth_at(t). Verified on blocker: the
wall now goes red and GROWS as the sensor finds it (15 -> 28 -> 32 cells)
while the silhouette shrinks (45 -> 32 -> 28), with zero false ghosts,
which is correct for that story. This also makes the silhouette work for
movers, whose footprints were previously never in the static truth.
2. city_blocks had no streets between COLUMNS. Block size came from the row
pitch only, so the column spacing left blocks abutting and a 2x3 grid
rendered as two solid slabs. A scene whose whole point is choosing a route
through streets has to have streets. Both axes now get their own pitch and
gap; verified by connected-component count (9 rects -> 9 blocks, 6 -> 6).
Traces recorded before this carry no truth snapshots and fall back to the
story's static truth, so they still render.
Review caught it: in the pursuit scene the target never appeared to move. The
marker was placed once in build() at story.waypoints[-1] and nothing ever
touched it again, so a viewer saw a vehicle chase a stationary mark, drive
past it, and keep going.
The renderer had nothing to work with. The trace recorded goal_dist_m and
goal_index but not the goal's POSITION -- fine while every goal was a fixed
waypoint the story already declared, useless the moment one moves.
* fog_record captures goal_x/goal_y per tick from the active waypoint,
which the moving-goal path rewrites each step;
* fog_trace.goal_at() interpolates between ticks, for the same reason the
vehicle pose is interpolated -- a marker that steps once per world tick
judders at playback rates;
* fog_scene translates the marker and trails it in orange, so the chase
reads as a chase rather than as a vehicle wandering toward a mark that
keeps jumping. The trail is only shown once the goal has actually moved,
so the static stories are unchanged.
Measured on pursuit: the target travels 203.3 m and the gap closes from
149.0 m to 2.9 m.
tests: 5 new -- the target is recorded and really moves, the position
interpolates between ticks, and a static goal still reports a position so
there is one code path rather than two. 84 pass.
Two scaling defects, both invisible at fog-story size and both blocking the
city-scale scenes.
1. _edt2 called a scalar Felzenszwalb kernel once per row and once per column
through np.apply_along_axis: at 512^2 that is ~1024 invocations each looping
512 times in Python. Measured 3.85 s, and 83% of a scenario step -- against
belief.py's own docstring claiming 'the exact EDT at 512^2 is milliseconds',
off by three orders of magnitude.
Same algorithm, but the sweep is now the only Python-level loop and each
step works on every row at once:
128^2 0.217s -> 0.069s 3.1x
256^2 1.001s -> 0.174s 5.8x
512^2 3.851s -> 0.642s 6.0x
BIT-IDENTICAL, not merely close -- the field feeds a learned policy, and a
difference in the last ulp stops a run reproducing. Verified against the
scalar kernel across shapes 1..64 and densities 0..1, and by re-recording
three scenes and comparing trajectories exactly. Recording is ~2x faster
end to end (traffic 49.8s -> 23.3s).
2. Route inflation was hard-coded at 3 CELLS. Cells are not a fixed size: the
fog stories are 2.08 m/cell so that was the intended 6 m, but on a 1200 m
city raster at 256 the same 3 cells is 14.1 m -- wide enough to close every
street. Measured there: inflate 1 routes in 1847 m, 2 in 2006 m, and 3
finds NO ROUTE AT ALL, which is the difference between a demo and a
stationary vehicle.
Now specified in metres (inflate_m, default 6.0) and converted per raster.
Existing scenes are unchanged (still 3 cells); the same story at 1200m/256
gets 1 cell = 4.7 m.
tests: 10 new. The EDT oracle is the original scalar implementation, kept in
the test file so the two can never drift.
The design decision that matters: belief is PER AGENT. A shared map would be a
much smaller change and a much weaker demo -- the interesting behaviour is one
agent knowing about a wall another has never seen, so the two route
differently through the same city and only converge when their knowledge does.
Sharing a grid erases exactly that. Measured on the dispersal run: agent a had
seen 4593 cells and d 4096, with 1263 cells only a knew about.
Agents are also REAL to each other. Each one's sensor reads a truth grid that
includes the others' current footprints, so peers occlude, have to be
discovered, and get routed around -- the same treatment a Mover gets rather
than a special case. Nobody is told where anybody is. A peer the sensor just
saw is demoted into the decaying layer, or a moving agent would smear a
permanent wall along its path.
Implementation is deliberately thin: one FogScenario per agent over a shared
static truth, stepped in lockstep. That reuses the whole tested single-agent
path -- sensing, route spine, vehicle model, collision scoring -- instead of
forking it, and each agent's trace comes out in exactly the format the
renderer already reads, so a squad clip is N traces drawn into one scene
rather than a second format to keep in sync.
Two scenarios recorded and rendered:
rendezvous 4 starts -> 1 meeting point. All four arrive, zero contact,
465 ticks. They park on a 14 m RING rather than one cell:
converging four vehicles on a single point made them fight over
it and two never arrived.
dispersal 4 starts -> 4 opposite corners, so the paths cross in the
middle. Three of four reach (346.9 / 334.1 / 380.6 m, zero
collisions each); agent c stalls at 263.7 m without contact --
same family as traffic's looping, and worth fixing before it
ships.
The rendered fog is the TEAM's combined coverage, because one frame cannot
show four private maps; the per-agent sensor rings are what keep the
distinction visible.
tests: 10 new -- beliefs are distinct objects, knowledge genuinely diverges in
both directions, a peer lands in every OTHER agent's truth and never its own,
static truth is never mutated, agents step in lockstep, and a squad of one
behaves like the plain scenario (so this is a wrapper, not a fork). 120 pass.
Same simulator, same renderer, a real world: an OpenStreetMap + SRTM scene
bundle rasterized into the occupancy grid the belief machinery already
consumes. Two variants, which is the point of putting them side by side --
'without fog of war' is expressed as the agent simply STARTING with the true
map. Same sensor, same planner, different initial knowledge.
Measured on 1200 m of Austin at 384 cells (3.13 m/cell, 31.3% built):
steps map updates driven vs straight collisions
full map 2904 0 1780.4 m +8.9% 0
under fog 3022 618 1861.7 m +13.8% 0
Knowing nothing about the city costs 81 m -- 4.6% -- and no contact. That
comparison is the deliverable.
Three things had to be fixed to get there, each of which was silently wrong
rather than obviously broken:
* ENDPOINTS must come from the largest connected component of free space
UNDER THE PLANNER'S OWN INFLATION. Chosen from raw free space, a cell can
be free and still unreachable once the route is inflated, and a run drives
most of the way before reporting no route for the last stretch.
free_components/far_pair_in_free_space do this properly.
* THE GROUND PLATE was a constant tuned for the +-100 m stories. On a 1200 m
city it was smaller than the world, so the capture's crop calibrated onto
a patch in the middle and the clip showed a fraction of the map. It now
derives 16:9 from the story bounds; the original stories are unchanged.
* NO-FOG RENDERS LIT. Drawing fog tiers over a map the agent fully knows
would be a lie about what it can see.
build_scenario/record accept a rasterized truth and prior so a real city can
be supplied without stuffing a megabyte of occupancy into the frozen,
content-hashed Story.
The bundle path is a required runtime argument with no default and appears
nowhere in this repo: the DATA is OpenStreetMap (ODbL) and SRTM (US public
domain) so the renders are publishable, but the bundle is a local artifact of
a private project.
tests: 8 new, covering the parts that do not need the bundle -- component
labelling under inflation, the doorway that closes when inflated (the Austin
failure in miniature), endpoints landing in one component, a fully blocked map
raising rather than guessing, and plate sizing. 128 pass.
… path Review notes, addressed. DEAD AIR. The dispersal clip ran 168 s because one agent could not close its last stretch, holding the recording open to max_steps while everyone else sat still. Trimming is done at RENDER time, not by stopping the simulation: a stall cutoff in the sim also killed runs where agents were only briefly jammed crossing the centre and would have arrived (measured -- cutting on 90 stalled ticks ended the run at 318 ticks with NOBODY reaching, against 3 of 4 when left alone). The capture now ends shortly after the last arrival. 168 s -> 73 s with nothing interesting lost. GREEN WAS BORING. In the rendezvous it started on the right, close to the meeting point, and drove a straight line -- 97 m against 204 m for the others. Moved to the left side with blue and orange so all three have to thread the block grid, which is the thing the scene exists to show. THE AUSTIN COMPARISON IS NOW ONE CLIP. The full-knowledge path is drawn underneath the fog run in dim blue-grey, so the deviation is visible directly instead of requiring the viewer to remember the previous video. Presentation only -- reference_xy carries a path from an earlier run and never enters the simulation. Also: Squad.run gains stall_ticks for callers who do want to stop early; it is off by default for the reason above. 128 tests pass.
traffic drove 353 m to cover 152 m, turning 3.2 full circles on the way. The
cause was the local controller chasing a carrot only 14 m ahead on a route
that is replanned constantly by moving obstacles: every replan jogs the
carrot sideways and the controller over-steers after it.
route_lookahead_m is now a story knob, and traffic uses 22 m:
lookahead ticks driven turns collisions
14 m 739 353.4m 3.2 0
22 m 202 156.3m 0.7 0 <- 152.1 m straight line
30 m 184 157.4m 0.6 0
Two things I tried first and backed out, recorded so they are not retried:
* FORCING THE CARROT AHEAD of the vehicle (walk further along the route if
the subgoal lands behind). This removed the controller's ability to turn
around at all and produced 38 truth penetrations where there had been
zero -- strictly worse than the loop it was meant to fix.
* REBUILDING ONLY WHEN THE DYNAMIC LAYER CHANGES rather than whenever it is
non-empty. Correct in principle and kept, but nearly neutral in practice
(177 -> 171 replans, identical trajectory): with movers crossing the map
the layer genuinely does change almost every sense tick.
Other stories keep the 14 m default and are unchanged (ghost 143.9 m / 0.4
turns, city 228.2 m / 1.3 turns, both still zero collisions).
128 tests pass.
A camera placed by framing alone flies through buildings constantly on a real
map -- measured on Austin, 1156 of 2000 shots had their line of sight inside
geometry, and at a 3 degree ground-level angle it was 396 of 400. After this,
zero of 2000.
clear_eye solves for the smallest lift directly rather than searching: raising
the eye by d raises the point at parameter t by (1-t)*d, so requiring
z(t) >= height(t) + margin gives d = max over t of (height + margin - z)/(1-t).
Only the height changes -- sliding the shot sideways would fight whatever
framing decision put it there.
TWO THINGS THE MEASUREMENTS CAUGHT:
* Binning mesh VERTICES marks a building's corners and leaves its ROOF
HOLLOW: 2.5% of cells against a 31.3% footprint, so a ray over the middle
of a tower read height 0 and was judged clear. The corner heights are now
flooded inward across the footprint -- a grey dilation masked by
occupancy, which for extruded buildings reconstructs the roof exactly and
cannot leak outside it. A footprint island whose vertices all rounded into
neighbouring cells is never seeded (4359 such cells on Austin, each a
building the camera would fly through) and falls back to the median
building height: over-guessing costs altitude, under-guessing costs the
shot.
* The margin must apply over GEOMETRY, not over open ground. Requiring 12 m
of clearance everywhere forbade every low approach to a subject standing
in the open -- the camera has to come down somewhere. Caught by a test
asserting a clear shot is left untouched, which failed for exactly that
reason.
frame_group fits the group's bounding sphere from a fixed bearing (a camera
that also chases the group's heading swings wildly while they are still
converging), and SmoothCamera damps the result, with separate time constants
because a focal point that lags its subject reads as a mistake while an eye
that lags reads as weight.
tests: 11 new. 139 pass.
One renderer, two cameras per frame: a cinematic shot that frames every vehicle and a fixed overhead minimap composited into the corner. It cannot be two renderers — SceneGraph::setRenderer is single-attachment, so a second one over the same scene moves every actor to it and silently blanks the first (measured: mean luma 81.8 -> 0.0). The HUD is drawn in the scene as VTK 2-D actors rather than burned in by the encoder, which needs the renderer to reach Python as a live vtkRenderer. Text over a city of pale concrete is unreadable, so each line backs itself. Two markers per vehicle, because one cannot serve both shots: a beacon for the main view, scaled off the camera distance so it holds its size on screen rather than towering over a converged group; and a flat arrowhead held above the skyline for the overhead shot, where a beacon seen end-on is one dot. The minimap is fixed on the whole map, not a second tracking shot — "where are they" is only answerable against ground that stays put — and is cropped to the map's own aspect so no inset pixels go to empty sky.
clear_eye can only answer occlusion by raising the camera, and raising is
the wrong answer when the obstruction is next to the subject rather than
between: a stadium wall 30 m from the group pushed the eye 330 m up and
flattened the shot into a plan view. A camera operator would step sideways.
So search the bearings and score each on whether the VEHICLES are visible,
not on whether the line to their centroid is. Those differ exactly when it
matters — the centroid of a group straddling a corner sits in the open while
half the group is behind the wall. Lift and bearing deviation only break
ties, so the camera holds its scheduled angle whenever that angle works.
Measured over 240 frames of the four-agent rendezvous on Austin, counting
every eye-to-vehicle ray:
fixed bearing + lift 52.0% blocked, mean lift 69.2 m (max 330.4)
bearing search 20.2% blocked, mean lift 0.0 m
+ visibility scoring 9.3% blocked, mean lift 3.5 m
The lift going to zero is the point: it finds bearings that need no climb at
all, so the low angle survives. Mean bearing deviation is 2.1 degrees — it
sits where the schedule asks and swings out only when it truly cannot see.
Searching elevation too was tried and dropped: 3x the cost to move one ray
in 960. What remains is a vehicle driving hard against a wall, where the ray
grazes that wall in its last few metres; no camera position fixes it and
none needs to.
Also makes the tail exemption a distance rather than a fraction — 12% of a
600 m shot stopped checking 78 m short of the subject, which is a whole city
block — and gives the shot an opening high angle that eases down.
goal_dist_m is the distance to the nav's current sub-goal, which route following holds at one lookahead ahead. Measured over the eight-agent rendezvous: it read 13.9 m at tick 0 and never left 9.5-14.0 m across 1600 m of driving, so the HUD showed 'goal 14 m' from the far side of the city. The recorded goal position gives the real number, and it also stays correct when the goal is the moving kind.
The two acts were ad-hoc scripts, so the clip could be watched but not regenerated. They are now a module with the layout rules stated, plus a CLI command that records and renders both. The rendezvous is a staging LINE rather than a ring, and this is the whole reason act one works: a ring puts three of its eight slots on the far side, and since every agent enters from the west the first five to arrive park in a wall between the stragglers and their goals. Measured on the ring, the three agents holding the easternmost slots (x=518..540, against peers parked at x=399..471) spent the last minute orbiting at 10 m/s, ending 115 m, 121 m and 329 m short. Slot order now matches start order, so no agent crosses another's path. Act two starts exactly where act one ends, so the two clips are continuous. BUNDLE stays a required argument with no default — the geometry is not in this repository and must not be assumed.
vtkTextActor font sizes are absolute pixels, so a HUD laid out at 720p renders three-quarters the intended size at 900p — legible in every test render and small in the actual deliverable, which is the worst way for this to fail. Everything is now a fraction of frame height; at 720p the fractions reproduce the original 30/18/26/24 px design exactly.
The finale drew no goal markers at all, so the pursuit act would have been eight vehicles chasing nothing on screen — the same class of failure as the target marker pinned to the opening waypoint that the last review caught, arrived at from the other direction. Each agent now has a goal post seated at its recorded goal position, so it moves when the goal moves. Thinner and shorter than a vehicle beacon on purpose: a goal is a place, not a protagonist. Two vehicles share one target in the pursuit, so the posts are nudged around a small circle rather than z-fighting on the same cell. The pursuit also frames its targets together with the vehicles, so the gap closing is visible instead of implied. The rendezvous deliberately does not: its goals start a kilometre away and framing them would hold the whole map, and eight specks, for the whole clip.
Squad._stamp_peers wrote each agent's body into every other agent's truth_now. The first line of FogScenario._stamp_movers is `truth_now = truth.copy()`, and it runs microseconds later on every tick, so every peer footprint was erased before anything sensed against it. Measured with eight agents 17 m apart: 63 cells stamped, 0 surviving. Two claims were therefore false. Peers did not occlude and did not have to be discovered -- an agent drove through the squad as if it were empty air. And "zero collisions" was vacuous for peers, because truth_now is the grid penetration is scored against, so a collision between two vehicles could not be counted even in principle. Peers now arrive as a mask the scenario ORs back in after its own reset, which is order-independent and cannot be clobbered. Verified: 63 cells survive, and all eight agents have a peer in their decaying layer within 43 ticks. The test that should have caught this is why it survived. It called _stamp_peers and asserted truth_now immediately, before _stamp_movers ever ran -- verifying the write rather than its persistence, and passing green the whole time. It now asserts through the step that used to erase it, and a separate test drives two agents inside sensor range and asserts a peer is genuinely sensed. That fixture is separate on purpose: the stamping fixture starts its agents 120 m apart with a ~38 m sensor, which is fine for stamping and useless for discovery. Every recorded squad trace predates this and is superseded: trajectories and penetration counts will change now that peers are real.
Contributor
Author
Correction: the squad was mutually invisible (fixed in d6475ec)The PR description above claims peers "are real to each other: a vehicle
Two consequences:
Peers now arrive as a mask the scenario ORs back in after its own reset, which Why the test suite did not catch it. The test called Every recorded squad trace is superseded — trajectories and penetration counts |
Profiling the 8-agent Austin tick found 93.6% of 1375.5 ms in three functions, but neither hot one is slow for algorithmic reasons: the EDT issues ~92,000 numpy dispatches per build on 384-element vectors, and A* spends 17-22 us per expanded node on scalar numpy indexing, heap traffic and dict lookups. Both are dispatch tax, which is why the earlier 6x numpy vectorisation left the EDT still in first place. Records two dead ends so they are not retried. Incremental dirty-region EDT is ~1.4x, not ~1000x: a distance transform is global, and one obstacle cell in a 99.7%-empty optimistic map changed 281-384 of 384 rows. And CUDA loses here on measured numbers - the GPU is worth ~21 cores against 16 available, GPU A* is 6.5x SLOWER than a CPU heap A*, and D2H plus compute already loses to the 16-core CPU, so a GPU EDT with a host consumer never wins at any agent count. Crossover is ~60 agents at 384^2. Also records the fidelity hazards that make a fast twin worthless if missed: float32 EDT tie-breaks flip the gradient on medial axes and therefore the steering bias, and heapq's whole-tuple comparison has to be reproduced exactly or A* returns a different equally-optimal path.
Two complaints, one root cause. Framing eight agents spread across ~860 m through a 30-degree lens forces the camera to 4.67 group radii, and a fixed elevation turns all of that distance into altitude -- so the wider the squad spread, the higher the camera climbed. Exactly backwards. Three changes, measured over the real 856-frame rendezvous: eye above ground, mean 989 m -> 231 m eye above ground, max 1596 m -> 296 m view-direction rate, mean 29.6 -> 3.9 deg/s view-direction rate, max 84.5 -> 8.3 deg/s A 55-degree lens frames the same group from 2.40 radii instead of 4.67. A height ceiling picks the steepest elevation that keeps the eye under it, so distance becomes a longer flatter look across the city rather than altitude. And the bearing search now pans at a bounded 11 deg/s instead of snapping: it re-runs every frame, so a bearing it cannot reach this frame it simply chooses again next frame. Also caps the corrective climb, which was the real reason the camera was so high. clear_eye solves lift = max (need - z)/(1 - t), and making the tail exemption a DISTANCE put near_cut at 1 - 30/dist -- so on a 1000 m shot the last sample divides by 0.03 and a 40 m building beside the subject demanded a 1800 m climb. That fixed one bug and created another. The occlusion metric had to change with it. Scoring vehicle BODIES said 49.9% were hidden at the lower eye and drove the camera back up; but the renderer stands a beacon on every vehicle, and only 0.8% of BEACONS are hidden there against 0.0% at the old altitude. A body behind a building is what a low camera in a city looks like. Scoring chassis the viewer is not tracking made the hidden term saturate, swamp the lift and turn terms, and turn the bearing choice to noise.
Act two built a fresh SmoothCamera and restarted the shot schedule at u=0, so the join was a cut twice over: the damper had no history and snapped to whatever it was first handed, and the elevation jumped back to its opening establishing angle just as the squad broke north. capture_finale now takes and returns a CameraState, and renders a SUB-RANGE of one schedule rather than a fresh 0..1. The range is split in proportion to each act's length -- 178.5 s and 94.5 s, so the seam falls at u=0.654 -- which keeps elevation and bearing drift continuous in time across the cut. The damper does the rest. Act two wants a much wider frame than act one ended on, because it frames four northern targets alongside the vehicles; primed with act one's final eye it pulls back over about three seconds instead of cutting.
FollowGoal is a goal that IS another agent, resolved live. MovingGoal replays
a path fixed before the run starts; a convoy cannot, because where the vehicle
in front will be is not knowable in advance -- it is reacting to a world it is
still discovering. Squad binds them before the first step, and step order is
load-bearing: declared front to back, a follower reads its leader's position
after the leader has moved this tick.
Two behaviours fall out of machinery that already existed rather than being
scripted. A follower's target is also a peer, so it is stamped into the
follower's ground truth and has to be SEEN by a sensor ray like any obstacle --
nobody is told where it is. And because the target occupies its own cell, the
follower's goal snaps to the nearest free cell beside it, so the convoy closes
to a standoff instead of driving into the vehicle it is chasing.
Which navigator runs was previously unknowable and unselectable: use_planner
was a FogScenario constructor default, not a Story field, so every story
silently used the route spine and the reactive mode could not be reached at
all. It is now a Story field, recorded in the trace manifest, and named in the
HUD.
They are not interchangeable. Same seed, same local controller, only the route
spine differs -- driven distance against the straight line:
route+sdf sdf-only
ghost +20% -3% reactive is SHORTER
unit +13% -2% reactive is SHORTER
blocker +6% +237%
city +15% +116%
traffic +3% +252%
AUSTIN +14% never arrives (4500 ticks, 941 m, stuck)
On a map with one obstacle the route spine is a small tax. Once there is
topology or motion it is worth 2-3.4x, and on the real city the reactive
controller does not solve the problem at all. Zero collisions in every case,
both ways -- the difference is whether it arrives, not whether it is safe.
The convoy also needed the traffic scene's lookahead fix: at the 14 m story
default the leader oscillates, closing only 166 m of an 810 m run in 36 s
against 335 m at 22 m. Measured with the leader ALONE, so the followers were
never the cause.
The two navigators look identical in a still frame and are not remotely equivalent in what they can do, so the clip should not leave it to be guessed. Read from the trace manifest rather than inferred.
I edited three files to record the navigator in the trace manifest and ran black on two of them, so CI's black --check went red on the third. The lint step runs over the whole package; checking only the files I remember touching is not the same test.
…osition Three things, all reported from review. PARKING. A vehicle sitting on its goal pirouettes: gdir is (goal - p) / (dg + 1e-6), which is pure noise once dg is small, so the carrot whirls around the vehicle and the steering chases it. It now brakes and holds heading. Measured after arrival: 0.0 deg/s of heading change against a sustained spin, speed to 0.0002, and arrival itself is unchanged (ghost 256 -> 255 ticks, city 410 -> 409). Parking is COMMANDED by the scenario, not inferred from the goal distance. I tried inferring it first and it broke arrival: under a route spine the navigator's goal is the lookahead sub-goal, snapped to a free cell by _nearest_free, so "close to my goal" is true constantly while travelling and again a few metres short of the real waypoint. The vehicle braked on the snapped sub-goal, stopped 2.74 m out, and the run never completed. Only the scenario knows an objective is terminal. SPEED. AgentSpec.vmax overrides the story's single world-wide vmax per agent. A chase in which quarry and hunter move at the same speed never resolves either way. THE INSET. The heading arrows were visibly offset from their vehicles because the overhead camera was perspective and the arrows are held above the skyline so no roof can hide them -- being nearer the lens than the ground, each projected outward from centre by alt/(alt-z). At the 1506 m camera altitude and 350 m marker height that is 30 m of error at 100 m from centre, 121 m at 400 m, 169 m at 560 m: up to 50 inset pixels, which is why it read as a bug. The inset is now orthographic, where altitude cannot displace anything, and which is what a minimap should have been anyway.
PNG encoding was the frame cost, not rendering: measured 333 ms for the two 1600x900 encodes against 33 ms of actual GL work. frameRGB() hands back the same pixels without the encode/decode round trip, and leaves no directory of thousands of files behind. Measured end to end: 350 -> 226 ms per frame-pair. The precedent already existed in fog_capture.open_encoder; the 3-D path just never used it.
The 2-D clips have shown three-tier fog since the start -- never seen,
remembered, visible now -- and the 3-D ones showed none of it, which made them
look like a flythrough of a city rather than a record of what was discovered.
A terrain-draped decal now carries the RECORDED fov grid as an RGBA texture,
written in place through libcvc's zero-copy texture path (setTexture(zeroCopy)
+ texture_modified(), which has its own C++ aliasing test at
src/cvcGL/test/cvcgl_texture_zerocopy.cpp).
A texture rather than geometry, and the reason is specific to this playback
speed: at 0.25 world seconds per frame against a 0.24 s sensor cadence the
visible set changes EVERY frame, so the 2-D renderer's "re-mesh only when the
fov changed" optimisation buys exactly nothing here. Re-meshing the lit cells
measured ~172 ms/frame; the texture write costs 14 (226 -> 240 ms/frame-pair
end to end). The decal's own mesh is 129x129 and never changes -- all the fog
detail is texels, so the two resolutions are independent.
What is drawn is the UNION of the squad's ever_seen. That is coverage, which is
a real measured quantity and exactly what the recorded caption claims
("coverage shared, knowledge private"). A union of the agents' private
occupancy beliefs would be a map no agent holds, so it is not drawn.
Lab.add_mesh cannot carry UVs, so the decal geometry is built directly.
WALLS. The city mesh now carries a projected coverage texture, so a building the squad has looked at is lit and one it has not recedes into the dark. A ground-only fog decal left the whole 978k-face city fully lit above it, which read as a dark carpet under a revealed city -- the opposite of the point. A GLSL shader is reachable here (vtkOpenGLShaderProperty takes fragment replacements, and it was tried) but buys nothing this does not: world-XY texture coordinates generated once, one small RGBA texture rewritten per frame, measured free against the render. The texture domain is offset by half a cell plus a pad ring because the belief grid is a grid of POINTS while texels are AREAS -- without it the mask lands a whole cell off -- and the pad ring clamps the city outside the simulated box to "unseen" instead of smearing the border row across it. Walls are widened one cell: a ray stops at the first cell it hits, so only a block's near rim is ever marked and a big building would otherwise stay dark with a lit edge. The ground decal keeps the exact set. PATHS. Driven paths in per-agent hue in both shots; the belief-space route spine dim and ONLY in the overhead inset. That is deliberate. planner.plan() string-pulls the route in belief space before it is stored, so early in a run it is a single straight segment over a kilometre long -- honest, because the planner has not looked yet and genuinely believes it is clear, and it reads correctly on a 2-D map. In the 3-D shot it is a line through the skyline that reads as a rendering bug. Both live in one long-lived node each, replaced in place with setGeometry. Lab.add_path cannot be used per frame: addGraphics removes and rebuilds the node (7-14 ms a call) and leaks a signals2 connection into m_boundsConns on every re-add. setGeometry also resets the render mode, and a lines-only cvc::geometry reports SURFACE_TRI, so it needs an explicit setRenderMode(LINES) afterwards or it draws nothing at all. 226 -> 272 ms/frame-pair for all of it, against 350 before any of this landed.
The projected mask lit a whole building CELL, so a 90 m tower lit to its roof because a vehicle drove past its base. The simulator cannot do better on its own: its sensor is a 2-D cast over a 384x384 occupancy raster, which is all the planner ever needed and all it ever used, and it carries no height at all. A post-pass now replays the recorded positions and casts a real elevation fan against the actual 978,242-face mesh, keeping the FIRST face each ray hits -- so a facade the squad drove past lights up, its upper storeys do not, and a wall behind another wall stays dark. Measured on the rendezvous act: 5,904 of 978,242 faces ever seen (0.60%), growing monotonically from 1,466 at tick 200. It is a POST-PASS on purpose. The dynamics are untouched and no trace is re-recorded; the planner did not have this information before and does not have it now. It is a rendering input derived from recorded positions under a stated sensor model, and the deliverable should say so rather than imply the agents knew it. Two measurements shaped the implementation. vtkOBBTree costs 401 us/cast because IntersectWithLine collects EVERY intersection along the ray, which in a dense city is most of a block; vtkStaticCellLocator's first-hit form costs 21.8 us and builds in 0.2 s against 4.2 -- 18x, and the first hit is the only one occlusion cares about. And because first_seen is monotone, replay only ever adds faces, so the renderer walks a presorted order and touches just the faces lighting up this frame instead of recolouring all 978,242 every time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fourteen commits taking the fog-of-war demo from three toy stories to a squad
crossing a kilometre of real city, plus the fixes that review turned up.
What the demo can now show
visible are drawn differently, and real geometry the agent has not found yet
is outlined as a silhouette —
truth & ~believed, so the viewer can see whatthe agent is about to be surprised by.
do not wait to be reached. The navigator retargets in place, so the route
follows a moving goal rather than restarting each time it moves.
own map, route and SDF. Nothing is shared but ground truth, and they are real
to each other: a vehicle occludes and has to be seen, exactly like a wall.
buildings and SRTM terrain, and a 3-D finale with a solved cinematic camera,
a live in-scene HUD, and an overhead minimap.
Fixes that came out of review
not its position, so the renderer had nothing to move the marker with. Goals
are now recorded per tick and interpolated like the vehicle pose.
overflowing the ring drawn around it.
as already known, which is the one thing a fog demo must not do.
that arrives by event rendered amber ("believed but not real") while being
entirely real.
covers that in under two seconds. At 22 m: 353.4 m → 156.3 m driven against a
152.1 m straight line (+132% → +2.7%), 176 → 33 map updates, still zero
contact. Two other candidate fixes were tried and reverted — a carrot-ahead
guard caused 38 collisions, and rebuilding the dynamic layer only on change
was measurably neutral.
goal_dist_mis thedistance to the nav's current sub-goal, which route following holds one
lookahead ahead: it read 13.9 m at tick 0 and never left 9.5–14.0 m across
1600 m of driving.
The camera is solved, not art-directed
A camera placed by geometry alone flies through a real city constantly. Lifting
it clear is the wrong answer when the obstruction is beside the subject rather
than between — a stadium wall 30 m away pushed the eye 330 m up and flattened
the shot into a plan view. So the camera searches bearings each frame and scores
them on whether the vehicles are visible, not on whether the line to their
centroid is; those differ exactly when it matters.
Measured over 240 frames on Austin, counting every eye-to-vehicle ray:
The climb going to zero is the point — it finds bearings needing no climb, so
the low angle survives. Searching elevation as well was tried and dropped: 3x
the cost to move one ray in 960.
Performance
The distance transform is vectorised — 6x faster, bit-identical output.
Notes for the reviewer
not in this repository and must not be assumed to exist at any path.
grl-snam finale BUNDLErecords and renders the finale end to end.cstalls 263.7 m short. Not acollision and not a planner failure — the local controller gives up when the
route keeps changing underneath it.