Repository navigation
feat: decrypt encrypted BTHome and MiBeacon advertisements - #10
Merged
Merged
Conversation
Closes the README item "URL callback to Prometheus Push Gateway", and adds the
scraping route alongside it.
--metrics-addr 0.0.0.0:9184 serves /metrics for Prometheus to scrape.
--pushgateway-url http://gateway:9091 POSTs the same exposition text every
--push-interval seconds. Both can be on at once; they share one registry.
Scraping is documented as the default choice because it gives Prometheus its own
staleness handling and needs no extra moving part. The Pushgateway is what you
want when the Pi cannot be reached from the Prometheus server, which is the case
the README had in mind.
Exported per sensor, labelled {mac,name,format}: temperature_celsius,
humidity_percent, pressure_hpa, illuminance_lux, moisture_percent,
battery_percent, battery_volts, rssi_dbm, last_seen_timestamp_seconds, and the
counter mitempr_readings_total. A measurement no sensor reports produces no
series and no HELP/TYPE header, rather than an empty family.
Two details worth calling out:
- Measurements are merged into the previous snapshot rather than replacing it.
A MiBeacon sensor alternates between a temperature event and a battery
event, so replacing would make each series flap in and out of existence
between scrapes and turn every rate() over it into nonsense.
- Gauges keep their last value, so a sensor whose battery died would otherwise
report a plausible temperature forever. That is what
mitempr_last_seen_timestamp_seconds is for, and the README shows the
staleness query.
Label values are escaped, since a name comes from a config file and can contain
a quote or a newline that would otherwise produce a malformed scrape.
No new dependencies. The exposition format is nine metric families of plain
text, and the HTTP surface is one GET route, so both are written by hand against
tokio's TCP rather than pulling in the prometheus crate and hyper for it. The
consequence is that the Pushgateway client speaks plain HTTP only, with no TLS:
that is documented, and the answer for a gateway behind TLS is a reverse proxy
or scraping instead. A Pushgateway that is down is logged and retried and never
takes the scanner with it.
Twelve tests cover the exposition text, the absent-measurement and
merge-across-advertisements rules, label escaping, the URL parsing including its
rejections, and three that drive the real request handler over a loopback socket
to check the status line, content type and framing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes the last README item, "also decode **encrypted** data".
A sensor set to encrypt gets its bind key from its config block:
[[sensor]]
mac = "A4:C1:38:00:11:22"
bindkey = "231d39c1d7cc1ab1aee224cd096db932"
The advertisement is then decrypted before the object parser sees it, so
everything downstream -- output, --exec, Prometheus -- works unchanged.
Both formats use AES-128-CCM with a 4-byte MIC, but they differ in ways that
matter and that are easy to get wrong:
BTHome v2 nonce = MAC || D2 FC || device-info byte || 4-byte counter
no associated data
MiBeacon nonce = MAC reversed || product id || frame counter
|| 3-byte counter
associated data = 0x11
Note the MAC goes in one way round for BTHome and the other way round for
MiBeacon. Both constructions follow the reference implementations, cited in the
module docs: bthome-ble and xiaomi-ble.
Without a key an encrypted advertisement is still skipped rather than parsed --
that behaviour arrived with the two decoder fixes and is what the existing tests
check. A wrong key fails the MIC check and is reported. This is worth stating
plainly: a mistake in the nonce layout cannot produce plausible-but-wrong
measurements, because the MIC would not match. It shows up as "could not
decrypt".
Dependencies: aes and ccm from RustCrypto. Both are pure Rust with no C to cross
compile, which matters for the musl targets.
I have no encrypted sensor to hand, so this is not confirmed against a real
device. What the ten tests do check is a full round trip: they build a complete
encrypted advertisement and then decrypt it, so encryption and decryption derive
the nonce independently from the finished packet and a layout mistake fails.
Three of them go through handle_service_data() end to end and assert the
measurements that come out; others check that a wrong key and a MAC from a
different device are both rejected, and that bind keys are validated with a
useful message. Still worth a run against a real encrypted sensor before this is
trusted.
Threading the MAC and the key into the decoders needed a parameter, so
handle_service_data() now takes a DecodeOptions. It implements Default, i.e.
"no MAC, no key", which is the unencrypted case, so the change at the existing
call sites is mechanical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the last README item, "also decode encrypted data".
The advertisement is decrypted before the object parser sees it, so everything downstream — output,
--exec, Prometheus — works unchanged.Both formats use AES-128-CCM with a 4-byte MIC, but they differ in ways that are easy to get wrong:
D2 FC‖ device-info byte ‖ 4-byte counter0x11Note the MAC goes in one way round for BTHome and the other way round for MiBeacon. Both constructions follow the reference implementations, cited in the module docs: bthome-ble and xiaomi-ble.
Without a key an encrypted advertisement is still skipped rather than parsed — that behaviour arrived with the two decoder fixes earlier in the stack. A wrong key fails the MIC check and is reported. Worth stating plainly: a mistake in the nonce layout cannot produce plausible-but-wrong measurements, because the MIC would not match. It shows up as "could not decrypt".
Dependencies:
aesandccmfrom RustCrypto. Both pure Rust with no C to cross compile, which matters for the musl targets.handle_service_data()end to end and assert the measurements that come out; others check that a wrong key and a MAC from a different device are both rejected. Still worth a run against a real encrypted sensor before trusting it — that is partly why this one is last and a draft.Threading the MAC and key into the decoders needed a parameter, so
handle_service_data()now takes aDecodeOptions. It implementsDefault("no MAC, no key", the unencrypted case), so the change at existing call sites is mechanical.I went through the README TODO list and found a few real bugs on the way, so this
is one of ten PRs. They are meant to be reviewed and merged in this order, and
each is a separate branch so you can take some and leave others:
chore/ci-deps-and-metadata— CI, unused deps, lockfiletest/decoder-assertions— make the decoder tests actually assertfix/bthome-object-lengths— BTHome parser desync + encryption flagfix/mibeacon-frame-control— MiBeacon hardcoded offset + encryption flagfix/continuous-scan— one reading per sensor ever, and the watchdog loopfeat/logging-and-output(draft) — log levels,--format jsonfeat/config-file(draft) — name, calibrate and filter sensorsfeat/exec-hook(draft) —--execfeat/prometheus(draft) —--metrics-addrand--pushgateway-urlfeat/encrypted-payloads(draft) — AES-CCM bind keys6–10 are drafts because they build on each other; GitHub can only point a
cross-fork PR at
main, so until the ones before it are merged each diff alsoshows its ancestors. I will rebase and force-push as things merge, so each
shrinks to its own change. Nothing here is urgent — take them at whatever pace
suits, or tell me to close some.
Everything is checked with
cargo fmt --check,cargo clippy -D warningsandcargo testin arust:1.96container, sincebluer'sbluetoothdfeature isLinux-only and I develop on a Mac.