Skip to content

Let a run interrupt the link without killing the fleet - #4

Merged
HanielUlises merged 1 commit into
mainfrom
experiment/link-outage
Sep 28, 2026
Merged

HanielUlises merged 1 commit into
mainfrom
experiment/link-outage

Conversation

@HanielUlises

Copy link
Copy Markdown
Contributor

A fleet that stops answering is not the same thing as a fleet that has stopped, and until now the experiment could only produce the second. run.sh kills the adapter; it comes back believing its robots are at their chargers, so every task it accepts afterwards never finishes, and a mission that replans into that is not recovering from an interruption — it is failing against a different fleet.

Publishing false on /eplansys/rmf_link drops what arrives over the websocket from then on. The socket stays up, the frames are still counted, the requests still go out, and what is lost is the road back. true reads it again. Both transitions are logged by the bridge.

What it shows

experiments/oe4-desconexion/run-corte-reversible.sh, cut at 10 s into the drive, 20 s of outage, bridge waiting 30 s:

the link with the fleet is down; the fleet is not, and what it says from here on is dropped
goto_site: 0 of 1 RMF task(s) done after 30s, last underway
(goto_site scout):0 failed: RMF task timed out
the policy failed; replanning from the state the mission reached
the link with the fleet is up again; what it says from here on is read
planning (attempt 2 of 3)
mission complete

ok   (Kw scout contaminated) holds
ok   (Kw relay contaminated) holds
ok   (Kw observer contaminated) does not hold
ok   somebody on the team was spoken to: relay
ok   observer was spoken to by nobody

The first attempt dies because what the fleet said was dropped on arrival. The second plans from the state the mission reached, submits a fresh task to a fleet that never went away, and finishes. Five of five.

The check that had to change with it

mission_check gains heard_any, and the one-site launch asks with it instead of naming scout. Which team member goes to the site and which is told is the planner's choice — relay-dirty_relay_scout and relay-dirty_scout_relay both answer the goal — and a replan from a different state picked the other one, failing a mission that had succeeded. What the mission claims is that the finding reached the team over a channel the observer was not on, and that is now what is checked.

heard_something keeps working and its default is now empty, since which agent is spoken to belongs to a launch rather than to the node.

Checked

  • the reversible outage run above, five of five
  • the two-site fleet mission unchanged by this: 52.3 s, eight of eight, both robots still driving together
  • linters clean over the three packages

Publishing false on /eplansys/rmf_link drops what arrives over the
websocket from then on. The socket stays up, the frames are still
counted, the requests still go out, and what is lost is the road back,
which is what an outage is: the adapter is alive and the robots are
where they were.

Killing the adapter measures something else. It comes back believing its
robots are at their chargers, so the tasks it accepts afterwards never
finish, and a mission that replans into that is not recovering from an
interruption but failing against a different fleet.

With the outage reversible, the replanning the mission does carries it
through: the drive times out while the link is down, the mission replans
from the state it reached, and the second attempt finishes on a fleet
that never went away.

mission_check gains heard_any, and the one-site launch asks with it.
Which team member goes to the site and which is told is the planner's
choice, and a replan from a different state can pick the other one; what
the mission claims is that the finding reached the team over a channel
the observer was not on.
@HanielUlises
HanielUlises merged commit 09a3556 into main Sep 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant