Control plane: stop engine instances wedged across consecutive reclaim sweeps

[micahtyong]

Summary

Reclaim trainer instances whose idle models repeatedly fail to unload, so wedged engines eventually release their GPUs. Split out of #39.

Failed reclaim sweeps persist strikes in the engines key store. Strikes are at least three hours apart by default, matching the current backend operation timeout: rapid placement retries must not kill a healthy trainer during a long step. Any successful unload, live model, or new placement clears the strikes.

Before stopping, the engine atomically checks that its models still match the failed unloads and enters draining mode to reject new placements. Engines without the new endpoint are left running. Cancellation failures retain instance records and strikes so cleanup can retry; an already-missing Modal call can be cleaned up.

Safety review

• Fixed rapid retries counting as independent evidence of a wedge.
• Fixed new placements racing with a stale reclaim snapshot; strike changes during unload also invalidate that sweep.
• Fixed cancellation errors discarding tracking for an engine that might still be running.
• Rebased onto main at aa6e16e, retaining model-creation error handling.

Validation

• 189 control-plane, engine, Modal platform, and KV tests passed; two SDK tests excluded because they import Linux-only socket options on macOS.
• Regression coverage includes rapid retries, recovery, live models, new placements during unload, cleared strikes, draining over HTTP, and cancellation retries.
• Package build and diff whitespace checks passed.
• Full local suite could not run: missing PyTorch and existing Linux-only socket imports. Linux CI passed: 715 tests passed, one skipped, and package build succeeded. Codegolf CI also passed.

Link to original Devin session: https://modal.devinenterprise.com/sessions/8d6804dd317c41329d77eb2c427dda6f
Requested by: @micahtyong

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access that start with 'DevinAI' or '@devin'.

• Disable automatic comment, CI, and merge conflict monitoring