Initial tests to target base device functionality

[bruno-f-cruz]

Note: all the tests target the core registers / functionality ONLY for now
Note2: Considering the testing device to not be connected to an external synchronizer. This is important for tests like R_HEARTBEAT and R_SECONDS

Status after #37. Boxes are ticked where the verify suite has a check. Items marked prerelease run only with --prerelease. Some items are reworded to describe the check that exists, where the original assertion proved wrong against real firmware or was removed from the specification.

General

• Assert all core registers exist and return from a read request
• Assert that all registers have the correct data-type
• Assert that read-only registers return error on write, partial: probes exist on R_WHO_AM_I and R_TIMESTAMP_MICRO, and on R_TAG, R_VERSION and R_HEARTBEAT under prerelease, but not on the deprecated version registers
• Report round-trip latencies
• Report % messages dropped when sending back to back
• Check if 0-length messages are allowed, needs clarifying: a Read request already carries no payload and is exercised by every read, so the untested case is a zero-payload Write
• Check if scheduled messages are supported
• Check if error messages are returned when wrong dtype is written, untested: the read-only probes deliberately send correctly typed payloads so they test read-only enforcement rather than type validation

Additional per Register

R_<**>_VERSION

• Assert values match R_VERSION, prerelease, and the source of the core versus protocol non-conformance

R_ASSEMBLY_VERSION

• Assert equals 0

R_TIMESTAMP_SECOND

• Assert Write sets a new timestamp

R_OPERATION_CTRL

• Can toggle between modes Standby and active
• Set to active mode, listen for R_HEARTBEAT for a couple of seconds, prerelease
• Set DUMP and check if ALL core registers are returned at least
• Set MUTE_RPL and check if no reads are returned. Clear MUTE_RPL and test again
• Clear R_HEARTBEAT, Set ALIVE_EN check if R_SECONDS emits an event (this is optional, but we can assert if the feature exists or not)

R_RESET_DEV

(Not sure what to test here, it seems a bit hard to test this from the client??)

• Now covered by four checks: readable, exactly one boot provenance bit reported with every command bit clear, and two read-only bit probes under prerelease which found a second non-conformance

R_DEVICE_NAME

• Not sure (?), covered by readability and length checks

R_CLOCK_CONFIG

• Report REP_ABLE and GEN_ABLE
• CLK_UNLOCK can write to seconds
• CLK_LOCK cannot write to seconds, worth prioritising: without it a locked device can fail the R_TIMESTAMP_SECOND write check and be reported as non-conformant when it is behaving correctly

R_SERIAL_NUMBER

• Check it is readable. The duplication of the first two bytes of R_UID was removed from the specification, so that assertion was dropped

R_TIMESTAMP_OFFSET

• Assert it reads "0", prerelease. The not-writable half was dropped, since the specification marks the register writable

R_UID

• Check if 0 (likely not implemented), prerelease

R_TAG

• Check it is readable, exactly 8 bytes, and rejects writes, prerelease. Zero is not asserted, since R_TAG carries the firmware git hash and a populated value is conformant

R_VERSION

• Interface hash is read and reported, not asserted to be zero, prerelease, since a device that declares a schema legitimately reports a non-zero value

R_HEARTBEAT

• See R_OPERATION_CTRL, prerelease
• Check it returns 0 since all tests run without a synchronizer connected for now, the premise does not hold: the heartbeat is driven by the device clock and keeps firing with the synchronizer disconnected, so it does not read zero
• Check IsActive if Active via OPERATION_CTRL, prerelease

[bruno-f-cruz]

Closing this issue for now since #37 has been merged. Future tests may be added, but will be tracked on a single-issue basis.