Bound stream execution and validate formation/resource configuration #73

Current outcome

Make GoBatch safe under sustained producer pressure without unbounded processor goroutines or hidden queue growth. Enrichment and outcome collection need predictable admission, not CPU-based resource guesses.

Replanned on 2026-09-06 against GoBatch master 63ef757 and ShitQuant's recorder, enrichment, paper/replay, and flow workloads. This is an implementation target, not a claim that the behavior already exists.

Required behavior

• Define explicit finite active-batch, buffered-item and batch-size safety limits, with named documented defaults and a deliberate migration policy for existing unlimited behavior.
• Acquire execution capacity before creating work goroutines; keep intake/formed batches bounded while saturated. Account for the forming/formed batch in addition to active batches and configured buffers.
• Make saturation backpressure and admission cancellation explicit; stopping a saturated collector must not leave an internal sender blocked forever.
• Separate formation targets from hard safety limits. MaxTime/linger is not an end-to-end execution deadline, and dynamic formation updates cannot disable safety bounds.
• Validate invalid durations, contradictory settings and unsafe limits before use. Document zero/default meanings instead of silent contradictory clamping; maintain chosen legacy behavior only through an explicit documented compatibility path.
• Do not claim byte-accurate memory limits for arbitrary T, user handlers, or error-collection helpers. Drop estimated-memory tracking and system-resource-derived defaults from this scope.

Verification and completion

Prove active/retained-work bounds under sustained pressure, varying batch sizes, cancellation while saturated, EOF remainder and invalid config. Use barriers/counters and race tests rather than timing-only sleeps. State the retained-item bound and caller obligations.

Follow the repository's formatting, race-test, vet/lint, package documentation, example and changelog requirements for the changed surface. Report the actual supported behavior and migration; do not treat a passing coverage percentage as proof of these outcomes.

Scope and relationships

Coordinates with #95 lifecycle, #75, #84 and #90. Owns saturation/admission and validated configuration; provider rate limits, request units, retries and durable queues stay outside the stream core. Historical resourceTracker is design history only.

Design history

The earlier report/prototype remains below for provenance; the requirements above supersede conflicting prescriptions. Existing discussion is preserved.

type ResourceLimits struct { /* max active batches, max estimated memory, ... */ }
func DefaultResourceLimits() ResourceLimits          // sensible defaults from system resources
func (r ResourceLimits) Validate() error

type Options struct {
    Config         batch.Config
    ResourceLimits *ResourceLimits
}
func (o *Options) WithDefaults() *Options
func NewWithOptions(opts *Options) *Batch            // recommended constructor when using limits

// Example
limits := batch.DefaultResourceLimits()
b := batch.NewWithOptions(&batch.Options{
    Config:         batch.NewConstantConfig(&batch.ConfigValues{MinItems: 10, MaxItems: 100}),
    ResourceLimits: &limits,
})