Skip to content

feat: record why a verification failed, not just that it did - #6

Merged
oxyc merged 1 commit into
mainfrom
feat/classify-verification-failures
Jul 31, 2026
Merged

oxyc merged 1 commit into
mainfrom
feat/classify-verification-failures

Conversation

@oxyc

@oxyc oxyc commented Jul 31, 2026

Copy link
Copy Markdown
Member

The problem

The decision log flattened every rejection into missing or invalid — the least useful place to lose detail. An expired challenge and a forged one produced the same log line, but they mean opposite things: one says our validity window is too short for the form, the other says we're being attacked.

That gap had a real cost. A tester reported the Snellman consumer-service form rejecting a submission with no field highlighted. The two candidate causes — an expired challenge and a replayed payload — have identical symptoms, and there was nothing in the log to tell them apart.

The change

Challenge::classify() surfaces the reason the altcha-org/altcha library already computes and we were throwing away:

reason meaning
ok passed
missing no altcha field posted — bot, or the widget script never ran
malformed present but not a decodable payload
expired solved correctly, but the window had closed
invalid_signature not a challenge we issued (forged, or post-key-rotation)
invalid_solution our challenge, wrong answer
replay valid, but already accepted once (enforced in Integration)

verify() keeps its bool contract and defers to classify(), so there's a single parsing path rather than two that can drift.

For expired challenges the log also records expired_for — seconds past expiresAt. Ten seconds over versus twenty minutes over is the difference between tuning the window and looking somewhere else entirely, and it's how we'll tell whether the widget renewal in #5 is holding up in the field.

Context stays non-PII: a form id, a reason, a duration.

Verification

Observed end-to-end through the real Logger hook:

valid:                  altcha: pass {"form":45}
expired 90s ago:        altcha: fail {"form":45,"reason":"expired","expired_for":92}
forged (other secret):  altcha: fail {"form":45,"reason":"invalid_signature"}
garbage:                altcha: fail {"form":45,"reason":"malformed"}
missing:                altcha: fail {"form":45,"reason":"missing"}
replay (same twice):    altcha: pass  →  altcha: fail {"form":45,"reason":"replay"}

7 new unit tests, including the expired-vs-invalid distinction this exists for. 63/63 pass, pint --test clean.

Note

No behaviour change to what gets accepted or rejected — only to what's recorded. The add-on was already offered to Gravity Forms' logging framework; it just has to be enabled under Forms → Settings → Logging. Version bump rides along in #5.

🤖 Generated with Claude Code

The decision log flattened every rejection into `missing` or `invalid`,
which is the least useful place to lose detail. An expired challenge and
a forged one are the same line in the log, but they mean opposite things:
one says our validity window is too short for the form, the other says
we're being attacked.

That gap cost us. A tester reported the Snellman consumer-service form
rejecting a submission with no field highlighted, and there was no way to
tell from the log whether the challenge had expired or the payload had
been replayed -- the two candidate causes have identical symptoms.

Challenge::classify() now surfaces the reason the altcha-org library
already computes (expired / invalid_signature / invalid_solution) instead
of discarding it, plus missing and malformed. verify() keeps its bool
contract and defers to classify(), so there's a single parsing path.

For expired challenges the log also records `expired_for` -- seconds past
expiry. Whether a visitor was ten seconds over or twenty minutes over is
the difference between tuning the window and looking elsewhere, and it
shows whether the widget's renewal is holding up in the field.

Context stays non-PII: a form id, a reason, a duration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@oxyc
oxyc merged commit aa6e456 into main Jul 31, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant