Skip to content

Losing AdGuard is silent by design once #125 lands, and no existing rule would catch it #126

Description

@Gerrrt

#125 chose to have Unbound forward to AdGuard Home with the public upstreams also listed, so a dead AdGuard drops out of Unbound's forwarder selection and resolution continues unfiltered.

That's the right trade for the household — but it's worth naming what it costs, because it isn't free: it converts a loud failure into a silent one. Under Option A a dead AdGuard took the whole house offline, which is self-reporting within about ninety seconds. Under Option C the only symptom is ads gradually reappearing, which nobody reports, and which looks identical to a blocklist that simply doesn't cover a given site.

This is the same shape as #90, with one difference that makes it tractable: a quiet IDS is indistinguishable from a stopped one by construction, whereas a DNS server can be asked a question. So this is fixable, and it doesn't need a new mechanism.

What wouldn't catch it

  • The containers.rules.yaml family — ContainerRestartLoop, ContainerOomKilled — needs a metrics agent on the Winterfell mini PC. That's the Deploy Alloy to Saruman and oracle #88 pattern and the box doesn't exist yet (ADR-0008, ADR-0008: the sensitive-tier services and two new firewall rules are untracked #102).
  • InstanceDown covers the host, not the container, and only once the host is a target at all.
  • Nothing in network.rules.yaml touches DNS.
  • Most importantly: a probe through the normal resolver path would always pass. Query pfSense, get an answer, everything looks healthy — because the fallback is doing its job. That's precisely the trap, and it's why this needs stating before someone writes the obvious rule.

The mechanism already on the roadmap

#91 adds blackbox-exporter. Its dns module covers this directly, and this is a good argument for that issue's scope:

  1. Liveness — probe AdGuard directly, at its VLAN 99 address on port 53, not through pfSense. This tests the thing that actually matters (does it answer queries?) rather than whether a process is listed, and bypassing the fallback is the whole point. This is the check that closes the issue.
  2. Function — resolve a known-blocked domain and assert the answer is the blocked response, not a real address. This separates "AdGuard is up" from "AdGuard is up and its blocklists loaded", which is a distinct and quieter failure: a blocklist update that silently stopped working leaves a perfectly healthy-looking service filtering nothing.

Both are scrapes and rules — nothing new to install beyond #91 itself, and no agent needed on the mini PC.

Severity

Low. Nothing is down, nobody is blocked, and no data is at risk — losing this costs ad filtering until someone notices. It should route as a normal notification rather than urgent, which fits the existing alertmanager.yaml routing without a new receiver.

The reason to file it anyway is that "unfiltered" is a state the estate is now designed to enter silently, and an alert is the only thing that distinguishes "AdGuard is fine and this site just isn't on a list" from "AdGuard has been dead for three weeks."

Follow-up

Fold into #91 when blackbox-exporter lands, or keep separate if that ships first. Either way the ADR text from #125 should mention that graceful degradation is what creates this detection requirement — the two decisions belong together.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions