You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add lightweight observation hooks for bounded batch execution #70
Expose the information an application needs to size its queues and diagnose quota/latency problems, through one small dependency-free observation contract.
Replanned on 2026-09-06 against GoBatch master 63ef757 and ShitQuant's recorder, enrichment, paper/replay, and flow workloads. This is an implementation target, not a claim that the behavior already exists.
Required behavior
Report batch formation cause, actual size, queued/in-flight counts, queue delay, handler duration and distinct error categories.
Specify callback concurrency, blocking, panic handling, event ownership/lifetime and ordering guarantees. No unbounded internal telemetry queue.
Keep application payloads out of default events; applications adapt events to their preferred logger/metrics system.
Measure disabled and enabled overhead. Replace the old literal zero-overhead promise and competing Logger/StatsCollector APIs.
Establish reusable event vocabulary for request and flow runtimes without making their correctness depend on an instrumentation framework.
Verification and completion
Use synchronized formation/execution scenarios to assert useful events and counts, callback failure behavior, and bounded memory; benchmark optional-hook overhead.
Follow the repository's formatting, race-test, vet/lint, package documentation, example and changelog requirements for the changed surface. Report the actual supported behavior and migration; do not treat a passing coverage percentage as proof of these outcomes.
Scope and relationships
Depends on #73's bounded-execution state/timing contract. This is not a built-in logging backend, telemetry exporter, dashboard, or statistics database.
Design history
The earlier report/prototype remains below for provenance; the requirements above supersede conflicting prescriptions. Existing discussion is preserved.
Original issue: Optional logging & statistics (observability)
Summary
Add optional, zero-overhead-by-default observability to GoBatch: pluggable logging and runtime statistics/metrics. This captures work from two abandoned branches so it isn't lost.
Motivation
Today there's no way to see what a Batch is doing or to measure throughput/error rates without wrapping processors by hand. Two prior explorations tackled this from different angles; this issue tracks reconciling them into one design.
A. Pluggable Logger
Two candidate interface shapes were prototyped — pick one (or support both via adapters):
The batch would log: processing start/completion, source-read progress, per-batch details, and errors as they occur. A logging processor wrapper was also prototyped (processor/logging.go).
B. StatsCollector / runtime metrics
From feature/logger-stats @ e9a7c6f (batch/stats.go):
Decide minimal-Printf vs. leveled Logger (or ship minimal + a leveled adapter).
Source (for recovery — branches are being deleted)
feature/logger-stats (local commit e9a7c6f, ~2,100 LOC incl. LOGGING_STATS.md, batch/logger.go, batch/stats.go, processor/logging.go, processor/stats.go) — the fuller proposal; remote was already deleted, so this commit existed only locally.
Current outcome
Expose the information an application needs to size its queues and diagnose quota/latency problems, through one small dependency-free observation contract.
Replanned on 2026-09-06 against GoBatch master
63ef757and ShitQuant's recorder, enrichment, paper/replay, and flow workloads. This is an implementation target, not a claim that the behavior already exists.Required behavior
Verification and completion
Use synchronized formation/execution scenarios to assert useful events and counts, callback failure behavior, and bounded memory; benchmark optional-hook overhead.
Follow the repository's formatting, race-test, vet/lint, package documentation, example and changelog requirements for the changed surface. Report the actual supported behavior and migration; do not treat a passing coverage percentage as proof of these outcomes.
Scope and relationships
Depends on #73's bounded-execution state/timing contract. This is not a built-in logging backend, telemetry exporter, dashboard, or statistics database.
Design history
The earlier report/prototype remains below for provenance; the requirements above supersede conflicting prescriptions. Existing discussion is preserved.
Original issue: Optional logging & statistics (observability)
Summary
Add optional, zero-overhead-by-default observability to GoBatch: pluggable logging and runtime statistics/metrics. This captures work from two abandoned branches so it isn't lost.
Motivation
Today there's no way to see what a
Batchis doing or to measure throughput/error rates without wrapping processors by hand. Two prior explorations tackled this from different angles; this issue tracks reconciling them into one design.A. Pluggable
LoggerTwo candidate interface shapes were prototyped — pick one (or support both via adapters):
Minimal, dependency-free (from
cursor/implement-backwards-compatible-logger-hooks-3373@a58bbf8) — compatible withlog.Logger,slog,zap.SugaredLogger, etc.:Leveled (from
feature/logger-stats@e9a7c6f, local-only):The batch would log: processing start/completion, source-read progress, per-batch details, and errors as they occur. A logging processor wrapper was also prototyped (
processor/logging.go).B.
StatsCollector/ runtime metricsFrom
feature/logger-stats@e9a7c6f(batch/stats.go):Plus a stats processor wrapper (
processor/stats.go).Design notes / open questions
Batch[T]) on adoption.Printfvs. leveledLogger(or ship minimal + a leveled adapter).Source (for recovery — branches are being deleted)
feature/logger-stats(local commite9a7c6f, ~2,100 LOC incl.LOGGING_STATS.md,batch/logger.go,batch/stats.go,processor/logging.go,processor/stats.go) — the fuller proposal; remote was already deleted, so this commit existed only locally.cursor/implement-backwards-compatible-logger-hooks-3373(commita58bbf8,batch/logger.gominimal interface).