Skip to content

fix(cron): schedule validation and serialized SQLite access - #122

Open
adriannoes wants to merge 6 commits into
developmentfrom
fix/cron-schedule-and-sqlite
Open

adriannoes wants to merge 6 commits into
developmentfrom
fix/cron-schedule-and-sqlite

Conversation

@adriannoes

Copy link
Copy Markdown
Collaborator

Summary

Consolidates three cursor high-severity cron/gateway fixes as operator-authored commits:

Consolidation notes:

Testing

  • make test_cron — passed (overflow + partial-number + serialization tests)
  • GATEWAY=1 make test_gateway_db_lock — passed
  • CI=true make test — all 46 test binaries passed
  • make static — passed

Supersedes #104, #105 and #111.

The vendored SQLite build is SQLITE_THREADSAFE=0, and the gateway HTTP
thread shares that connection with the main loop. Dashboard memory,
session list, and cron routes, plus cron_poll's due-job query, touched
g_db without agent_lock. A dashboard refresh during a turn or a cron
poll is undefined and can crash the process or corrupt memory.db.

Supersedes #104.
An interval whose seconds do not fit in now+sec wrapped next_run into
the past. Delivery ack recomputed the same sum, so the job stayed due
and the agent ran it on every poll.

Supersedes #105.
strtol and strtoll accepted a numeric prefix, so at:2026-10-08 stored
next_run 2026. That one-shot was already due, ran on the next poll, and
ack deleted it. Comma lists such as 1,3,5 matched only the first value
and still scheduled.

Supersedes #111.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context. ShellClaw is a C99 edge agent (Jetson / Raspberry Pi) with SQLite memory/cron (SQLITE_THREADSAFE=0, one shared g_db), a libwebsockets dashboard thread, and a main-loop/channel thread serialized by a non-recursive agent_lock(). This PR consolidates #104 (serialize dashboard + cron SQLite), #105 (interval overflow) and #111 (partial-number schedules) into one branch.

I built test_cron with CC=gcc and linked it with -lm -pthread -ldl (this image has no libcurl). It passes. I could not run GATEWAY=1 test_gateway_db_lock here (no libwebsockets).

Summary. The cron parsing fixes are correct and well tested. The locking fix is incomplete: it adds agent_lock around most dashboard SQLite calls but misses one, DELETE /api/sessions/{id}. That leaves the same data race the PR sets out to close, and the new test cannot see it.

Must Fix

  • src/gateway/routes.c:509-517 handle_session_delete still calls session_delete(id) with no agent_lock(). It runs on the lws thread against the same g_db as agent_run and cron_poll. dispatch.c:50-55 (/reset) already locks this exact call, so the dashboard path is the only unlocked session_delete caller. agent.h now says dashboard g_db handlers must hold the lock, and the PR title says "serialized SQLite access", so this is a gap against the stated contract. Fix: wrap it like the sibling handlers (rc = session_delete(id) under lock, then build the response after unlock).
  • tests/test_gateway_db_lock.c: add DELETE /api/sessions/<id> to the route list. It would fail today. The probe lives in memory.c, but session_delete has no memory_sqlite_entry_probe() call, so the hook cannot observe it either. Add the probe there, otherwise the test passes while the bug is present. Also consider probing every g_db entry point (session_load, session_save, memory_save, cron_job_update_next_run) instead of a hand-picked subset, so a future unlocked caller trips the test.

Should Fix

  • CHANGELOG.md [Unreleased] / Fixed has no entry for any of the three fixes (dashboard/cron SQLite serialization, interval overflow, partial-number schedules). Recent consolidations (#119 to #121) had the same gap.
  • Behavior change for already-stored jobs: parse_field and interval_next_run now reject input that used to parse (interval:60abc, 0 9 1,15 * *, at:2026-...). cron_ack_delivery (cron.c, parse failure branch) silently moves such a recurring job to now + 365d. A job someone created with a comma list before this PR will stop firing for a year with no log line. Either log the job id and schedule on that fallback, or note the migration in the CHANGELOG. Rejecting comma lists at creation is right; the point is the existing rows.
  • cron.c cron_pick_due_row and the dashboard handlers now hold agent_lock while doing SQL. agent_run holds that same lock for a whole LLM round trip, so during a long agent turn GET /api/cron, /api/memory and /api/sessions block the single lws service thread, which also stalls WebSocket and /health. /asap already behaves this way, so this is consistent with existing design, but it makes the dashboard visibly freeze. Worth a doc line, or a follow-up to give SQLite its own mutex instead of reusing the agent lock.
  • handle_cron_create holds the lock across cron_create_job, which for a cron expression can scan up to 3662460 minutes with localtime_r per step. Parse/validate first and take the lock only for cron_job_create. Minor on desktop, but it is measurable on a Pi.

Nice to Have

  • at: still accepts ts <= 0 (e.g. at:-5, at:0), which is due immediately. Consider rejecting ts <= 0 the way interval: rejects sec <= 0.
  • parse_field still uses strtol, which accepts leading whitespace and +/- signs (+5, 5). Harmless after the *end check, but a digit-only check would match the strictness the comments describe.
  • test_schedule_rejects_partial_numbers covers at:, interval: and comma lists but not cron:0 9 * * 1-5x or cron:*/5 * * * * (step syntax is also rejected now, via the same path). A one-line assert for step syntax documents that it is unsupported.
  • memory_sqlite_entry_set_hook_for_test is compiled into production builds, like the two existing test hooks. Fine for now, but a #ifdef SHELLCLAW_TEST would keep it out of the shipped binary.
  • Makefile: test_cron now links agent.o, skill.o, config.o, toml.o and provider_common.o, and the test defines a stub provider_router_active_backend_snapshot. That is a heavy link for a cron unit test, and the stub will go stale if the agent adds more router symbols. Acceptable, but it is the first thing to break when agent.c grows.

Positive Highlights

  • interval_next_run is small and correct: errno == ERANGE, end check, sec <= 0, and sec > LLONG_MAX - now as a pre-check rather than a post-hoc overflow test. Folding #111 into the #105 refactor avoided two overlapping edits of the same parse path.
  • cron_pick_due_row releases the lock before the re-offer sleep and keeps the early-return paths balanced (agent_unlock() before return 1, and once on the fall-through). I found no lock leak or double unlock, and row is a heap-owned copy so using it after unlock is safe.
  • Tests assert the real outcomes rather than just return codes: the overflow test checks that ack lands at now + 365d, the parse test checks next is untouched on failure, and both lock tests check the mutex is released afterwards. The wsi == NULL guard in dispatch_route that makes the route testable without a socket is a clean small change.
Open in Web View Automation 

Sent by Cursor Automation: Code review

@adriannoes adriannoes changed the title fix(cron): schedule validation and serialized SQLite access (supersedes #104, #105, #111) fix(cron): schedule validation and serialized SQLite access Oct 16, 2026
@adriannoes adriannoes self-assigned this Oct 16, 2026
DELETE /api/sessions/{id} ran session_delete() on the lws service
thread with no agent_lock(), leaving the exact dashboard-vs-agent
SQLite race this branch sets out to close (SQLITE_THREADSAFE=0, one
shared g_db). Every sibling dashboard g_db handler already holds the
lock, and handle_message /reset locks this same call.

Wrap the session_delete() call in agent_lock()/agent_unlock() like
the sibling handlers and build the HTTP response after unlock, so the
route honors the contract documented in agent.h.

The route is now exercised by test_gateway_db_lock: seed a session
with session_save() and DELETE it through dispatch_route. That
requires session_delete to call memory_sqlite_entry_probe(), matching
the other probed entry points; without it the hook cannot observe the
call and the test would stay green while the bug is present. Verified
red first (probe present, lock missing: FAIL g_sqlite_unlocked),
then green after the lock.
The parse-failure branch of cron_ack_delivery moves a recurring job to
now+365d without a trace. Since the stricter schedule parser (interval
overflow, partial-number rejection), a stored job created with a now
invalid schedule (e.g. a legacy comma list) would silently stop firing
for a year after its first delivery.

Log the job id and schedule on that branch, house style (shellclaw:
cron:), before the row is freed (cron_job_row_free clears row.schedule,
so it stays ahead of the branch). Fail-close behavior is unchanged.
Document the three consolidated fixes under Unreleased/Fixed: dashboard
and cron SQLite serialization including DELETE /api/sessions/{id},
interval-overflow rejection, and partial-number schedule rejection.
Also note the stored-jobs behavior change: a stored schedule the stricter
parser rejects now snoozes +365d on ack and is logged, so the gap is
diagnosable instead of silent. Reviews of the sibling consolidations
called out the same changelog gap.
@adriannoes

adriannoes commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator Author

Review disposition (all Must Fix / Should Fix / Nice to Have items):

Must Fix

  • Unlocked session_delete() in handle_session_delete (DELETE /api/sessions/{id}), and the missing memory_sqlite_entry_probe() in session_delete: Fixed in 9347780 fix(gateway): serialize session deletion with the agent lock. The call is now wrapped in agent_lock()/agent_unlock() like the sibling handlers (response built after unlock); session_delete calls memory_sqlite_entry_probe() like the other probed entry points; DELETE /api/sessions/<id> added to tests/test_gateway_db_lock.c (seeds via session_save, asserts 200). Verified red first: probe present + lock missing → FAIL: tests/test_gateway_db_lock.c:42 g_sqlite_unlocked == 0; green after the fix.
  • "Also consider probing every g_db entry point (session_load, session_save, cron_job_update_next_run)": Not changed beyond session_delete. The current hook set covers the races this PR fixes plus the new session-delete test route; expanding probes further is a follow-up as remaining entry points gain test routes.

Should Fix

  • CHANGELOG.md [Unreleased] / Fixed gap: Fixed in 97a4a92 docs(changelog): record the cron schedule and SQLite serialization fixes — bullets for the dashboard+cron SQLite serialization (including session deletion), interval-overflow rejection, partial-number rejection, and the stored-jobs snooze behavior.
  • Silent 365-day snooze for stored jobs with now-rejected schedules: Fixed in c91904d fix(cron): log the job snoozed on a rejected stored schedule — cron_ack_delivery now logs job id and schedule (shellclaw: cron: house style) in the parse-failure branch; fail-close behavior unchanged. The changelog entry also notes the behavior change for legacy rows.
  • Dashboard freeze during long agent turns / dedicated SQLite mutex: Not changed. Dashboard g_db handlers holding agent_lock() matches the existing /asap design this PR extends; an own-mutex redesign is a scoped follow-up, not a patch on this branch.
  • handle_cron_create lock scope (validate before the lock): Not changed. Lock-scope micro-optimization; hoisting validation means splitting cron_create_job's contract (parse + insert) shared with the cron tool path, so it is not a tiny clearly-correct change here. Durable fix is the dedicated SQLite mutex follow-up.

Nice to Have — Not changed, all five: at: ts <= 0 rejection, digit-only parse_field, step-syntax test assert, #ifdef SHELLCLAW_TEST for the probe hook, test_cron link weight. Incremental hardening outside this PR's smallest correctness patch — follow-ups.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant