Repository navigation
feat: config file for naming, calibrating and filtering sensors - #7
Merged
Merged
Conversation
Adds a CI workflow, removes dependencies that are declared but never referenced, and fills in the package metadata. No behaviour change. CI: .github/workflows/rust.yml was deleted in f3575ab, so nothing checks the code. The new ci.yml runs fmt, clippy -D warnings and tests, plus a separate job that cross-compiles both Pi targets so releases cannot break silently between tags. Dependencies: hex and anyhow are not referenced anywhere in src/. Narrow tokio from "full" to the four features actually used, drop bluer's "full" (the mesh/L2CAP/RFCOMM stacks are unused), and drop uuid's "v4" feature, which pulls in rand for a generator the code never calls. All three cut compile time and ARM binary size, per the README's Pi Zero W goal. Cargo.lock: mitempr is a binary crate, so the lockfile belongs in version control -- it makes a build from a tag reproducible and turns dependency updates into a reviewable diff. Clippy: &Vec<u8> -> &[u8] (ptr_arg) and inlined format args, so that -D warnings is green. Callers are unchanged. Cross.toml: every line is commented out, and the dockerfile paths it would enable are matched by "Dockerfile.*" in .gitignore, so it cannot work as written. Release workflow: replace `cargo install cross` (compiles from source on every job) with taiki-e/install-action@cross, drop the redundant strip step since [profile.release] already sets strip = true, add fail-fast: false and workflow_dispatch, pin the toolchain, and publish a .sha256 per asset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three existing decoder tests call handle_service_data() and drop the
return value, so they pass no matter what the decoders produce. Hand-decoding
the fixtures they already contain gives concrete expected values, so assert
those instead:
BTHome 0x40 00 12 01 64 02 7D 09 03 8D 18
-> battery 100 %, 0x097D = 2429 -> 24.29 degC,
0x188D = 6285 -> 62.85 %
PVVX MAC A4:C1:38:A0:7B:03, 0x08F2 = 2290 -> 22.90 degC,
0x1919 = 6425 -> 64.25 %, 0x091D = 2333 -> 2.333 V, 0x10 = 16 %
MiBeacon frame control 0x2050, product 0x01AA, event 0x100D
-> 0x00EA = 234 -> 23.4 degC, 0x0261 = 609 -> 60.9 %
Also adds coverage for the paths that had none: BTHome voltage, negative
PVVX temperatures, the MiBeacon temperature-only/humidity-only/battery
events, and the four rejection paths (short PVVX packet, MiBeacon frame with
no event, unknown MiBeacon event id, unrelated or absent service UUID).
Five further tests are #[ignore]d with the reason spelled out, because they
describe packets the decoders currently get wrong:
- BTHome: an unknown object id advances the cursor by a guessed two bytes,
which desyncs the parser on a 3-byte object such as illuminance (0x05) --
the parser then reads a value byte as an object id and invents a humidity
reading of 5.12 % while dropping the real temperature.
- BTHome: the device-info byte is skipped unread, so the encryption flag in
bit 0 is never seen and ciphertext is parsed as plaintext objects.
- MiBeacon: the event offset is the constant 11, which only holds when the
MAC-included bit is set and the capability-included bit is clear. Frames
with a capability byte, or without a MAC, decode to garbage or not at all.
- MiBeacon: the frame-control encryption bit (bit 3) is never checked.
Ignoring them keeps the suite green while documenting the defects; the
follow-up fixes remove the #[ignore] attributes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two defects in the BTHome v2 decoder, both of which can turn a valid
advertisement into invented measurements.
1. Unknown object ids desynchronise the parser.
The cursor advanced by a guessed two bytes for any object id the match did
not handle ("Try to skip an assumed Type + 1 byte value to continue"). BTHome
objects are 1-4 bytes wide, so on anything else the cursor lands inside a
value and reads a value byte as the next object id. For example, illuminance
(0x05, 3 bytes) followed by a temperature:
40 05 E8 03 00 02 7D 09
decoded as humidity 5.12 % with no temperature at all: 0x03 from the middle
of the illuminance value was read as a humidity object, consuming the real
temperature object as its value. Correct is 10.00 lux and 24.29 degC.
Replaced with a length table covering the object ids in the BTHome
specification, including the length-prefixed text/raw objects and the
command/dimmer events whose width depends on the event type. Objects the tool
does not report are now skipped by exactly the right number of bytes instead
of derailing everything after them, and a genuinely unknown id stops the parse
with a warning rather than guessing.
2. The device-information byte was never parsed.
Byte 0 of the service data carries the BTHome version in bits 5-7 and the
encryption flag in bit 0. The decoder skipped it with a hardcoded `i = 1`, so:
- an encrypted payload was parsed as though it were plaintext objects,
happily reporting whatever the ciphertext happened to look like;
- a non-v2 payload was decoded with v2 rules.
Both are now rejected with a warning. Decrypting encrypted payloads needs a
per-device bind key and is a separate change.
Also removed BTHOME_V2_PREAMBLE, which was prepended to the payload and then
sliced straight back off, and made a payload that yields no measurement at all
return None rather than a reading with every field empty.
While here, added the three-byte pressure and illuminance objects, the
single-byte humidity/moisture objects and the 0.1-resolution temperature
(0x45) to SensorData, since the length table has to know about them anyway.
SensorData now derives Default so adding a field does not touch the other
decoders.
The two #[ignore]d BTHome tests from the previous commit now pass and have
had the attribute removed. Nine new tests cover pressure, illuminance,
single-byte humidity/moisture, 0x45 temperature, correct skipping of an
unreported object, stopping at an unknown id, truncated objects, non-v2
payloads and object-less payloads.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Xiaomi decoder located the data object with a hardcoded constant:
// The byte at index 11 is the Type Identifier byte
const TYPE_IDENTIFIER_OFFSET: usize = 11;
11 is the length of frame control (2) + product id (2) + frame counter (1) +
MAC (6), which only holds when the MAC-included bit of the frame control is
set and the capability-included bit is clear. Both are optional per the
MiBeacon format, so:
- a frame with a capability byte has its object one byte later (two more if
the capability byte announces an IO capability). The decoder read the
capability byte as the event id and gave up.
- a frame without a MAC has its object six bytes earlier, and the decoder
read a value byte as the event id -- either giving up or, when that byte
happened to look like a known event, reporting a fabricated measurement.
The offset is now computed from the frame-control bits. Alongside that:
- The event id is read as the little-endian u16 it is (0x100D, 0x1004, ...)
rather than just its low byte, so an event whose low byte collides with a
known one can no longer be mistaken for it.
- The object's length byte is honoured instead of assumed, which makes the
payload self-delimiting: an event this tool does not report is skipped by
its announced length, so the events after it are still found. Previously
only the first object was ever looked at.
- Frames with the encryption bit (frame control bit 3) set are rejected.
They were being parsed as plaintext, which produces believable-looking
rubbish. Decrypting them needs a per-device bind key and is separate work.
- Frames with the object-included bit clear carry no measurement at all and
are rejected rather than parsed.
Bit positions and offsets cross-checked against the MiBeacon format as
implemented by https://github.com/Bluetooth-Devices/xiaomi-ble.
Since the loop now walks every object, illuminance (0x1007, three bytes, lux)
and moisture (0x1008, one byte, percent) are decoded too -- they reuse the
fields added for BTHome, so it is one match arm each. LYWSDCGQ's existing
events (0x1004, 0x1006, 0x100A, 0x100D) behave exactly as before.
The three #[ignore]d MiBeacon tests now pass and the attribute is gone, so the
suite has no ignored tests left. Seven new tests cover the IO-capability byte,
a frame without the object bit, several objects in one frame, skipping an
unknown object without losing the ones after it, a truncated object, and the
illuminance and moisture events.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Continuous scanning did not actually work. Two things stood in the way.
1. The seen-devices gate.
let mut seen = seen_devices.lock().await;
if !seen.contains(&addr) {
seen.insert(addr);
handle_device(...)
}
handle_device() therefore ran exactly once per device and every later
advertisement from that sensor was dropped, until BlueZ happened to emit
DeviceRemoved. One reading per sensor per BlueZ device lifetime is not a
continuous reader.
2. discover_devices() only announces each device once.
Since the watchdog timestamp is refreshed only on a successful decode, and
decodes had effectively stopped after the first pass, the watchdog starved and
fired every --watchdog seconds. Restarting discovery makes BlueZ re-announce
its cached devices, which produced another round of readings -- so the tool
appeared to work, but only because it was tearing discovery down and building
it back up every 20 seconds. That is the "get this darn thing to be more
responsive (#bluez)" item in the README.
Switched to discover_devices_with_changes(), which re-emits DeviceAdded
whenever a device's properties change, and removed the gate. Readings now
arrive as the sensors broadcast them and the watchdog is fed by real traffic,
so it stops restarting discovery in a loop and goes back to being what it was
meant to be: a recovery path for a wedged adapter.
Replacing the gate with nothing would report the same reading repeatedly,
because BlueZ also signals property changes that carry no new advertisement
(an RSSI update, for instance). The service data of the last advertisement is
therefore remembered per device and an unchanged one is skipped. Sensors bump
a packet counter on every broadcast, so a genuinely new reading always differs.
Also: handle_device() made three sequential D-Bus round-trips per event
(name(), rssi(), service_data()). Now that it runs on every advertisement
rather than once per device, that cost is paid constantly, so it uses a single
all_properties() call instead -- one round-trip for all three. On a Pi Zero W
with a handful of sensors nearby this is the difference between a few dozen
D-Bus calls a second and a third of that.
Finally, dropped two log lines that only made sense when each device was
touched once: the raw per-UUID service-data hex dump, and "Unknown BLE packet"
for every phone, watch and TV in range. Both now fire per advertisement and
bury the actual readings. A verbosity flag to bring the raw dump back is
separate work.
Not verified on hardware in this branch -- needs a run on a Pi with sensors in
range to confirm the readings repeat and the watchdog stays quiet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Readings were only ever printed as a Debug dump:
println!(" 🔍 Got sensor reading: {decoded:?}");
which is fine while poking at the code and useless to anything downstream.
Nothing could consume the output, and every message was unconditional, so
there was no way to quieten a running daemon or to ask it for more detail.
Logging
-------
println!/eprintln! are replaced with log + env_logger, so messages have a
level and a timestamp. -v turns on the debug messages (why a payload could not
be decoded), -vv adds the raw service-data bytes -- restoring the hex dump that
became too noisy once every advertisement was processed rather than only the
first per device -- and -q leaves only warnings and errors. RUST_LOG still
overrides all of it, so a systemd unit can be made chattier without its command
line changing.
Per-packet decode failures log at debug rather than warn on purpose: a sensor
broadcasts several times a minute, so a payload this tool cannot read would
otherwise repeat the same warning forever.
env_logger is taken without default features, which drops its regex-based
message filtering along with the regex and aho-corasick dependencies. The
"humantime" feature is kept for timestamps; note that it pulls in jiff. Happy
to drop it if you would rather have the smaller binary -- under systemd,
journald timestamps every line anyway, and --format json carries an explicit
timestamp field.
Output formats
--------------
--format text (default) writes one human-readable line per reading through the
logger. --format json writes one JSON object per reading straight to stdout, so
it is pipeable into jq without log decoration in the way, and is not silenced
by -q. Keys carry their unit (temperature_celsius, voltage_volts, ...) and
measurements a sensor does not report are omitted rather than serialised as
null, so a consumer can tell "not measured" from "measured as zero".
I did not add a separate ndjson format: for a stream of readings it would be
byte-for-byte identical to json, so it would be two names for one thing.
Structure
---------
main.rs was doing argument parsing, discovery, the watchdog, device handling
and printing. Split into:
main.rs argument parsing, logger setup, adapter setup, shutdown
scan.rs discovery loop, watchdog, per-device handling
output.rs the Reading record and how it is written
decoder.rs unchanged in purpose
SensorData gains a `format` field so a reading says which decoder produced it,
which the output needs and which also makes it visible when a sensor changes
firmware and starts advertising differently. This replaces BlePacketType, which
was carrying the same information in a second enum.
Shutdown
--------
Ctrl-C and SIGTERM now stop discovery instead of the process being killed
mid-scan, which matters because SIGTERM is what systemd sends. This is why
tokio's "signal" feature comes back.
Verified with cargo fmt --check, clippy -D warnings and cargo test in a Linux
container; the three new output tests pin the text line and the JSON shape.
Not yet run against real hardware.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes the README item "define sensors in a config file & filter defined
sensors".
--config takes a TOML file:
[general]
only_known = true
min_rssi = -90
[[sensor]]
mac = "A4:C1:38:00:11:22"
name = "Living Room"
temperature_offset = -0.3
humidity_offset = 1.5
Everything is optional; with no file the behaviour is exactly what it was.
Naming. A configured name replaces the advertised one, so a reading is labelled
"Living Room" rather than "LYWSD03MMC" -- which also disambiguates the case
where several identical sensors advertise the same model name.
Calibration. temperature_offset and humidity_offset are added to each reading
before anything sees it, which is how you line a cheap sensor up with a
reference thermometer. Both default to 0. Only measurements the sensor actually
reports are adjusted: an offset never conjures a reading that was not there.
Filtering. only_known reports just the listed sensors, which is what you want
when the neighbours' thermometers are also in range. It is checked against the
address before any properties are fetched, so an ignored device costs one event
and no D-Bus round-trip. min_rssi drops advertisements from the edge of range,
where the payload is more likely to be garbled; an advertisement whose RSSI
BlueZ does not report is accepted rather than dropped, since absent is not the
same as weak.
--only-known and --min-rssi override the [general] section so a systemd unit
can be adjusted without editing the file.
Unknown keys are rejected (serde deny_unknown_fields) instead of ignored, so
writing temp_offset when you meant temperature_offset produces an error rather
than silently doing nothing. Duplicate MAC entries and malformed MACs are
rejected with the offending value in the message, and a missing or unparsable
file names its own path.
main() now returns Box<dyn Error> so a configuration problem is reported and
exits non-zero rather than being unwrapped.
Fifteen tests cover the parsing, the two filters and their boundaries, the
override precedence, calibration including the absent-measurement case, and
each rejection path. Verified with cargo fmt --check, clippy -D warnings and
cargo test in a Linux container.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mreymann
marked this pull request as ready for review
August 4, 2026 17:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the README item "define sensors in a config file & filter defined sensors".
Everything is optional; with no
--configthe behaviour is exactly what it was.Naming. A configured name replaces the advertised one, so a reading is labelled
Living Roomrather thanLYWSD03MMC— which also disambiguates several identical sensors advertising the same model name.Calibration. The offsets are added before anything sees the reading, which is how you line a cheap sensor up with a reference thermometer. Only measurements the sensor actually reports are adjusted: an offset never conjures a reading that was not there.
Filtering.
only_knownreports just the listed sensors, which is what you want when the neighbours' thermometers are also in range. It is checked against the address before any properties are fetched, so an ignored device costs one event and no D-Bus round-trip.min_rssidrops advertisements from the edge of range, where the payload is more likely to be garbled; an advertisement whose RSSI BlueZ does not report is accepted rather than dropped, since absent is not the same as weak.--only-knownand--min-rssioverride the[general]section, so a systemd unit can be adjusted without editing the file.Unknown keys are rejected rather than ignored, so writing
temp_offsetwhen you meanttemperature_offsetproduces an error instead of silently doing nothing. Duplicate and malformed MACs are rejected with the offending value in the message, and a missing or unparsable file names its own path.main()now returnsBox<dyn Error>so a configuration problem is reported and exits non-zero rather than being unwrapped.Fifteen tests cover the parsing, both filters and their boundaries, the override precedence, calibration including the absent-measurement case, and each rejection path.
mitempr.toml.exampleand a README section come with it.I went through the README TODO list and found a few real bugs on the way, so this
is one of ten PRs. They are meant to be reviewed and merged in this order, and
each is a separate branch so you can take some and leave others:
chore/ci-deps-and-metadata— CI, unused deps, lockfiletest/decoder-assertions— make the decoder tests actually assertfix/bthome-object-lengths— BTHome parser desync + encryption flagfix/mibeacon-frame-control— MiBeacon hardcoded offset + encryption flagfix/continuous-scan— one reading per sensor ever, and the watchdog loopfeat/logging-and-output(draft) — log levels,--format jsonfeat/config-file(draft) — name, calibrate and filter sensorsfeat/exec-hook(draft) —--execfeat/prometheus(draft) —--metrics-addrand--pushgateway-urlfeat/encrypted-payloads(draft) — AES-CCM bind keys6–10 are drafts because they build on each other; GitHub can only point a
cross-fork PR at
main, so until the ones before it are merged each diff alsoshows its ancestors. I will rebase and force-push as things merge, so each
shrinks to its own change. Nothing here is urgent — take them at whatever pace
suits, or tell me to close some.
Everything is checked with
cargo fmt --check,cargo clippy -D warningsandcargo testin arust:1.96container, sincebluer'sbluetoothdfeature isLinux-only and I develop on a Mac.