Skip to content

uds: ignore a second tester's flow control, and send CFs in order - #26

Closed
tkeairns wants to merge 24 commits into
feat/autosar-ecufrom
feat/isotp-stray-fc
Closed

tkeairns wants to merge 24 commits into
feat/autosar-ecufrom
feat/isotp-stray-fc

Conversation

@tkeairns

@tkeairns tkeairns commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Draft, stacked on #18 (base feat/autosar-ecu; retarget to main once #18 merges). Split out of #18 because it fails the can-full Renode bench intermittently, see "Open" below. #18 is green without it.

What

  • zephyr/patches/zephyr/isotp-stray-fc.patch, applied by west patch apply when just updates the workspace (_west now hashes west.yml, zephyr/patches.yml and the patches). Zephyr v4.4.2 ISO-TP:
    • Sender: an FC is taken only in WAIT_FC once the FF or the block's last CF went out (the FF now counts in tx_backlog); any other FC is ignored (ISO 15765-2 9.8.3), and one CF is in flight at a time (the M_CAN priority queue sends equal identifiers by buffer index, not in queue order).
    • Receiver: an FC reaching a receive context is ignored instead of aborting the request being received or discarding one just received.
  • testing/suites/uds/test_dcdc_node.py: test_a_second_testers_flow_control_is_ignored and test_a_request_survives_a_second_testers_flow_control drive the node by hand with a second tester's FC (DLC 3, BS 0) after every frame of ours. The hand reader filters the node's frames by id only (on vcan every frame from the host reads is_rx=False).

Why

A second tester on 0x7E0/0x7E8 (a bench condition on the HIL bus) flow-controls the node's responses too. Unpatched, its FC released a CF with one in flight (CFs out of order on the wire) or aborted a request: the tester timed out (UDS P2 timeouts in the HIL soak).

How tested

Run Result
just twister with the patch 43/43 test cases
HIL (NUCLEO-H753ZI, can0 500 k, a second tester on the bus), 600 s coexistence soak x2 84 UDS cycles each, 0 UDS errors, DAQ complete, 0 CF breaks in 5462 node transfers; both fail only on bus_health (the other tester's controller errors, not the node)
HIL stray-FC tests, found image vs patched fail 2/2 vs pass
can-full CI (Renode), stray-FC tests fail without the patch (response cut short: 6 of 50 bytes), fail with any part of it alone (receiver hunk; sender FC rule; FF count + one CF in flight), pass with all of it

Open (why this is not in #18)

On the can-full Renode bench, with the whole patch, test_communication_control_stops_the_broadcast and test_communication_control_silences_the_pdus failed in 3 of 4 runs; without the patch or with any part of it they passed in 7 of 7. Both run before the stray-FC tests. Node-side logging of every UDS request (debug run) shows:

  • autosar: after 28 01 the node's clock ran 5.0 s while about 0.8 s of host time passed (the test sleeps 0.5 s), so S3 expired (tick sess 3->1) and the next 28 00 met the default session (NRC 0x7F). Renode advances idle virtual time faster than real time.
  • uds node: the node held communication = 1 (session 3) from the 28 01 on, yet DCDC_Status kept arriving on vcan for about 0.3 s.

So the failures are timing on the simulator, but the patch changes them from never to usually, and the mechanism is not proven. Needed before merge: a node-time check of the second case (frame times from the node, not the host) and either a test fix that holds in node time or a patch change.

…m it

arxml/can-full.arxml describes the can-full bench as an AUTOSAR R19-11
System Template: bench.dbc's frames unchanged, six new DC-DC frames on
0x310-0x315 (E2E profiles 1, 2 and 5, a container I-PDU with short headers,
a multiplexed I-PDU, a SecOC profile 1 secured I-PDU), UDS as DCM I-PDUs over
CanTp, XCP as general-purpose I-PDUs. It loads strictly, without warnings or
dangling references, in autosar-data and cantools.

tools/arxml2c.py generates the DC-DC's C tables from it: ids, timing, signal
layouts and scaling, E2E and SecOC parameters. `just arxml` runs it and the
firmware workflow fails when the tables and the ARXML disagree. A description
the core cannot send (big-endian signals, long or little-endian container
headers, unaligned SecOC fields, other E2E profiles, raw ranges beyond float
precision) fails generation rather than the node.
lib/autosar is a sans-IO core (signal packing, E2E profiles 1, 2 and 5 per
the PRS, the container short header, multiplexed segments, the SecOC
trailer) and its Zephyr side: one thread sends a schedule of PDUs on their
ARXML timing through lib/can, earliest deadline first and below the
application threads, and AES-128 CMAC comes from PSA Crypto (mbedtls and
tf-psa-crypto join the west allowlist).

The DC-DC maps each model sample onto the generated tables, one build
function per PDU; a PDU the ARXML adds without one does not compile.
CommunicationControl stops them with its other normal messages. nodes/full
takes the snippet, adding 200 frames/s (about 5 % of 500 kbit/s). The SecOC
key is RFC 4493's example key: public on purpose, a demo key.

full.elf grows by 10.8 KB of text and 13.3 KB of RAM: the 3 KB thread stack,
8 KB of AES tables, PSA state.
suites/autosar tests the DC-DC as a bench tests an AUTOSAR ECU, with
cantools reading arxml/can-full.arxml: ids, lengths and cycles as described;
E2E profile 1 by hand (cantools has none), 2 and 5 by cantools; the
container's PDUs in turn; the multiplexed PDU's alternatives; the SecOC MAC
verified with the key (cryptography); every value equal to DCDC_Status's of
the same sample; UDS on the CanTp addressing and flow control the ARXML
gives; 0x28 silencing the PDUs. It records the bus with candump, out of
process, and fails when the host dropped frames rather than blaming the node.
can-full runs it in CI and on hardware.

ztests check the core against the PRS E2E example tables, RFC 4493's CMAC
examples and signal scaling at its edges.

The UDS suite's self-test test now reads its socket during the 2 s routine:
at the added 200 frames/s the idle socket overflowed and dropped the final
response, which udsoncan had received.
On hardware the 600 s coexistence soak failed on one missing DAQ sample
while the node's counters were clean. With candump -d the record shows why:
the suite's candump, writing to a file, dropped 501 frames in four bursts,
while a second candump on the same run writing to tmpfs dropped none of
1.86 M frames and saw every DAQ sample. A write waiting on the disk stalls
candump until its socket overflows.

frames.recording() now has candump write into a pipe the test process
drains, records every frame when given no ids, and fails as an incomplete
record when the host drops frames instead of as a lost sample. The soak
records through it; its assertions are unchanged.
P0A94-00 DC/DC converter performance (overtemperature), P0A09-00 status
circuit low (input undervoltage), P0A08-00 status circuit (output
overcurrent), U0111-00 lost communication with the battery energy control
module (BMS_Limits missing). A scan tool shows the generic text instead of
manufacturer codes. Before: 0x1A0098, 0x1A0116, 0x1A0219, 0xC11187.
SAE J1979 services 0x01 (support bitmaps, MIL and DTC count, OBD standard,
the node's PIDs), 0x02 (freeze frame 0 from snapshot record 0x01 of the
first confirmed DTC), 0x03/0x07/0x0A (confirmed, pending, permanent DTCs as
2-byte codes), 0x04 (clear) and 0x09 (VIN from DID 0xF190, CALID, CVN, ECU
name) on the UDS identifiers. Unsupported requests get nothing, physically
or functionally. SAE J1979-2 on the same core: DIDs 0xF400 + PID, 0xF501,
0xF8xx; ReadDTCInformation 0x42, 0x55, 0x56; ClearDTC group 0xFFFF33.
CONFIG_ZELOS_UDS_OBD, on with the UDS server.

The DC-DC's PIDs are a four-row table, each a formula over one of its DIDs,
so the freeze frame reports them from the snapshot: 0x1F run time, 0x42
module voltage, 0x46 ambient temperature, 0x5B pack life.

New for every build: a DTC confirmed with its indicator requested is
permanent. ClearDTC and 0x04 leave it; it goes once its test has passed in
three consecutive operation cycles.
A scan tool's view of the DC-DC: single frames on 0x7DF, an ISO-TP stack
per responder for the flow control a multi-frame response needs. PIDs
walked from the bitmaps and checked against DCDC_Status; MIL and count; a
lost BMS pending, confirmed and in the freeze frame; a permanent DTC
surviving 0x04 and 0x14; the VIN by multi-frame; J1979-2 DIDs and 0x19
reports equal to the legacy services; silence to unsupported requests.

Its fault tests play the BMS, so beside the bench's BMS they skip; CI also
runs the suite on the image alone, as it runs can_raw.
DTCs follow 13 CCR 1968.2 (d)(2.2)-(2.5) on the one DTC memory UDS and OBD
share. An operation cycle is still one boot (any ECUReset ends it).

- Two-cycle confirmation: a failure stores pendingDTC and the freeze frame;
  confirmedDTC, warningIndicatorRequested and permanent follow when it fails
  in a second tested cycle (CONFIG_ZELOS_UDS_DTC_CONFIRM_CYCLES, 2).
- The MIL is warningIndicatorRequested: it stays on while the test passes and
  goes off after three passed cycles (..._HEALING_CYCLES), taking the
  permanent DTC with it. confirmedDTC then ages out after 40 cycles without
  a failure (..._AGING_CYCLES), one cycle being one warm-up cycle here.
- A permanent DTC whose MIL is off, after a clear or a power loss, goes after
  one cycle in which its test ran and never failed. core_init() takes the
  permanent set to restore after a power loss.
- Freeze frames: the first with the pending code (records 0x00 and 0x01, OBD
  freeze frame 0), the second (0xF0) at the first failure of each later
  cycle; erased with an unconfirmed pending code or the aged confirmed one.
  OBD freeze frame 0 belongs to the DTC stored first, not the lowest index.
- PID 0x01: MIL from warningIndicatorRequested; comprehensive components
  supported and complete. CONFIG_ZELOS_UDS_OBD_VIN: only one ECU reports
  the VIN.

J1979-2: DID 0xF810 = 0x01; ReadDataByIdentifier answers the DIDs it
supports and 0x31 only if none is (every DID, ISO 14229-1); 0x19 0x42
reports severity class 0x02 and applies the severity mask; 0x19 0x55
records are DTC and status, 4 bytes; 0x19 0x56 with no DTC in the group is
0x31; 0x19 0x04 records 0x00 and 0xF0 carry DIDs 0xF400 + PID; OBD requests
are never answered 0x13 or 0x21. Legacy: requests mixing availability and
data PIDs, more than 6 PIDs or 3 freeze frame pairs get nothing; 0x09 takes
up to 6 availability infotypes.

The uds suite's DTC test follows: pending before confirmed, the MIL kept
until three passed cycles, two failed cycles in extended data 0x01, three
snapshot records in 0x19 0x03, and a multi-DID read keeps the supported
DIDs. Renode zeroes RAM on a reset (its reset reloads the ELF), so the test
needing the memory to survive one skips there.
13 CCR 1968.2 (g)(4.4.6)(C): no power loss erases a permanent DTC. With
CONFIG_ZELOS_UDS_NVS (default with OBD-II and a storage partition) the
permanent DTCs are one Zephyr Memory Storage (ZMS) entry, their 3-byte
numbers, read at start and handed to the core, which restores them when the
RAM memory was lost. ZMS, not NVS: NVS takes sectors of 64 KB at most, and
the H753's erase sector is 128 KB. A work queue at the lowest application
priority rewrites the entry when the set changes: the flash driver
busy-waits, and nothing of the application, XCP or the server may wait on it.

The uds snippet moves the H753's storage partition to flash bank 2: a sector
erase stalls fetches from its own bank, and the code runs from bank 1.
Each new test names the clause it checks:
- a fault is pending in its first failing cycle, confirmed with the MIL and
  permanent in its second (13 CCR 1968.2 (d)(2.2.1)-(2.2.2), (d)(2.2.5));
- the MIL goes off after three passing cycles, the code stays confirmed and
  the permanent code goes with the MIL ((d)(2.3.2), (d)(2.4), (d)(2.5.1));
- a permanent code survives 0x04 and 0x14 0xFFFF33 and goes after one
  passing cycle ((g)(4.4.6)(C), (d)(2.5.2)(A));
- the freeze frame is stored with the pending code, and J1979-2 record 0x00
  carries it as DIDs 0xF400 + PID ((d)(2.2.7), (g)(4.3.1));
- 22 F810 = 0x01 (ISO 15765-4:2021 6.1.1), a multi-DID read answers the
  supported subset, malformed OBD requests get no 0x13 ((g)(3.4.2)(A)),
  19 56 of an empty group is 0x31, mixed availability and data PIDs get
  nothing (AUTOSAR SWS_Dcm_00943/00972).

0x19 0x42 and 0x55 are read with the suite's own byte layout (severity,
DTC, status; DTC, status), no longer through udsoncan, whose 0x55 parser
expects a severity byte. Operation cycles end with ECUReset; the fault tests
play or silence BMS_Limits, so they run against any DC-DC responder that
watches it, not only the node, and skip under Renode where a reset loses RAM.
The reset macro reloaded the ELF on every reset, including the firmware's
own reboot, and LoadELF zero-fills the RW segment that holds .bss and
.noinit. Every ECUReset under Renode was therefore a power loss: the DTC
memory kept in .noinit was erased, and the lifecycle tests that end
operation cycles with 0x11 0x01 had to skip there.

The ELF now loads once, when the machine is created. A machine reset keeps
memory but puts the vector table offset back to 0, where Renode's part has
an empty ITCM, so the CPU halts with SP and PC 0; the macro sets it to the
part's boot address (flash, 0x08000000) beside the SysTick frequency.

The obd suite's three cycle tests and the uds suite's operation-cycle test
no longer skip under Renode. Renode bench, full.elf alone: obd 15 passed (was
12 + 3 skipped), uds 60 passed + 4 timing skips (was 59 + 5). The can-full
bench is unchanged at 100 passed, 15 skipped: these tests silence the BMS,
and skip beside the bench's.
ISO 15765-4:2005 Table 7: the external test equipment sends one flow
control with block size 0 and STmin 0. The suite's scan tool sent
can-isotp's default block size, 8. Padding is the tester's choice; the
scan tool now pads with 0xCC, as the UDS client does.
ReadDTCInformation 0x1A of record 0x91 (in-use ratio) or 0x92 (test
results) answers 0x31: no DTC of the DC-DC has a ratio or test-result
monitor (comprehensive components only, 13 CCR 1968.2 (g)(4.5.1), (d)(5.1);
SWS_Dcm_01609), nothing functionally. 0x06 and 0x1A of those records are
OBD requests, so a malformed one gets no 0x13 and none gets 0x21.

The obd suite asserts that a J1979-2 scan tool reads what a legacy one does:
readiness (01 01 = F501 = F401), the support bitmaps, the DTC lists, and no
test results, ratios or PID 41 either way.
PID 42 is the live rail voltage; two reads at different times differed by
more than the bound under a loaded bench. F442 must lie in the span of the
PID 42 reads before and after it.
09 06 and 22 F806 computed the CRC-32 of the whole download window (64 KB
on the DC-DC) on every request: in any session, without security,
functionally, while a routine was pending, and under core_lock. The
control loop takes that lock on every DTC report, so each such request
could hold it off for the CRC's duration, with the UDS thread raised to its
priority meanwhile.

The CVN is now computed at start, and again on the first read after an
erase or a TransferData write. Both need the programming session with its
security level unlocked, so outside one no request computes a CRC; inside,
a read computes at most one per such change.
seal() took core_lock and returned, so an ECUReset right after a DTC
confirmation could reboot mid zms_write: a half-programmed flash word,
possibly an ECC error that fails zms_mount on every later boot, after
which permanent DTCs are never stored again.

seal() now finishes the queued or running write before taking core_lock,
which the write takes too, and then takes a lock the write holds through
the flash access, so a write begun in between is waited out and none
begins after. The reboot waits only for writes queued or begun before it.
… span

Frame 0's PID 5B was compared with itself whenever the pack voltage was
known; it is now checked against the snapshot's pack voltage, encoded as
the node does. F446 in a multi-DID read is live temperature: it must lie
within a degree of two PID 46 reads around it, not equal one taken later.

Dropped asserts other tests already make: F400 and F440 against PID 00 and
40, F40C's 0x31, and the silence of 06 00 and 09 08.
The memory window holds the last control cycle's state, so a read right
after the previous test restored the setpoint could still see 12.34 V.
The old check subtracted one second from PID 1F and failed on a responder
up for less than a second.
With CONFIG_ZELOS_UDS_NVS, a storage partition that does not mount, or a
write of the permanent DTCs that fails, sets P062F-00 (SAE J2012 internal
control module EEPROM error; 0x062F00, legacy 0x062F) through the normal DTC
path: it reports on 0x03, 0x0A and 0x19 0x42 and lights the MIL. AUTOSAR
NvM/Dem report a failed non-volatile block the same way, as an ECU-internal
DTC. The node runs on with its permanent DTCs in RAM, and the existing log
line stays. CCR 1968.2 (g)(4.4.6)(C) requires permanent DTCs to survive a
power loss; this is the ECU saying it cannot.

- Confirmed at its first failure: its pending state would live only in RAM,
  which a power loss takes with the store broken, so a two-cycle rule would
  never confirm it.
- Never written to flash: the mask the store holds and compares is the
  node's DTCs only, so its own permanent bit cannot retrigger the write that
  failed.
- In the core only from its first failure until it ages out or is cleared,
  as the node's last DTC (the glue appends it with zelos_uds_core_extend()),
  and kept across a reset (zelos_uds_core_kept()): while the store works,
  the supported DTC list and every report are unchanged.
- A mount, or a write that succeeds, passes it; it heals as any other.

ztest test_store_failure_dtc: P062F appended cleared, the node's DTC memory
kept, confirmed + MIL at one failure (0x03, 0x07, PID 0x01, 0x19 0x42), its
permanent bit, kept over a reset and passing with the MIL kept, and gone
after a power loss with a working store.
The same helper, now shared by the uds suites.
zms_mount() ran in zelos_uds_start(), on the main thread before the node's
threads: a blank storage partition (first boot, or a garbage collection cut
short) erases a 128 KB sector there, so CAN started 1-2 s late. The mount is
now the first job of the store's work queue, at the lowest application
priority, submitted once the UDS thread runs: the node starts at once.

- Until the mount, the permanent DTCs are those in RAM and requests are
  answered from them as always (no 0x21 on OBD); writes queue behind the
  mount, and a reboot waits out a mount in progress as it does a write.
- After a power loss, flash's permanent DTCs join when it has mounted
  (zelos_uds_core_restore(): applied only if the memory started cleared,
  as zelos_uds_core_init()'s argument was).
- A mount that fails sets P062F then, as before.

ztest test_permanent_dtcs_restored_after_the_mount: none before the mount,
flash's after it on a power-on, RAM's kept over a reset.
A second tester on 0x7E0/0x7E8 flow-controls the node's responses too.
Zephyr's ISO-TP (v4.4.2) took its FC, arriving while the FF or a block's
last CF was still pending, as the one due: it released the next CF with
one in flight, and the M_CAN priority queue sends equal identifiers by
buffer index, so CFs left out of order. An FC reaching the receiver
aborted the request being received, or discarded one just received.
Either way the tester timed out.

A west patch, applied when just updates the workspace, makes both sides
ignore an FC they are not waiting for (ISO 15765-2) and keeps one frame
in flight. Two HIL tests drive the node by hand with a second tester's FC
after every frame of ours.
python-can marks every frame from this host as not received, and on the
Renode bench the node is on this host: the hand reader saw nothing.
@tkeairns

Copy link
Copy Markdown
Contributor Author

Folded into #18 (feat/autosar-ecu, rebased on main): both commits are there as 43557f4 and e5e6d78, with a fix on top (3a29180: tx_backlog counted before can_send(), so a completion that beats its return cannot leave it at 1).

The CommunicationControl failures under Renode were not the patch: both tests waited bound(0.5) (2.5 s on a Renode bench) in a non-default session with no request, and a Renode bench catching up on time it fell behind runs up to ~2x the host, so S3 (5 s of node time) ended the session inside the wait. Reproduced on unpatched firmware with --time-scale 12; fixed in the tests (d0fd7d6, TesterPresent through the wait). Details in #18.

@tkeairns tkeairns closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant