Elect checkpoint-commit representatives by Modal task id

[micahtyong]

Summary

Multi-node Lilo checkpoints only ever contained node 0's shards, which makes every multi-node run unresumable. _sync_checkpoint_volume() elects one volume-committing representative per node by hostname:

dist.all_gather_object(hosts, socket.gethostname())
representative = hosts.index(hosts[dist.get_rank()]) == dist.get_rank()

Every Modal cluster container reports the same hostname. Verified with a 2-node @modal.experimental.clustered probe:

rank=0 hostname='modal' ips=['10.100.0.1', '10.100.0.2']
rank=1 hostname='modal' ips=['10.100.0.1', '10.100.0.2']

so hosts == ["modal"] * world_size, only global rank 0 is ever a representative, and ranks 8+ never Volume.commit() their .distcp shards. This is what killed lilo-27b-256k-pad-30-1789719362 (24 ranks, 3 nodes): checkpoint 000018 on lilo-checkpoints holds only __0_0.distcp … __7_0.distcp, and all nine chained resume attempts died with

FileNotFoundError: '/checkpoints/000018/…/miles/__16_0.distcp'

Fix: identify the node by MODAL_TASK_ID, which is distinct per cluster container and inherited by the Ray workers the container spawns (same probe: ta-01M330JFS8NWWBTRGYARPT93XR vs ta-01M330JFX4Y6528JDRY8D2EM6R, and a child process sees the same value). Falls back to the hostname off Modal.

Link to Devin session: https://modal.devinenterprise.com/sessions/ca503f6afb54412498a66f69af517adf
Open in Devin Desktop: https://modal.devinenterprise.com/desktop/session/ca503f6afb54412498a66f69af517adf?variant=devin
Requested by: @micahtyong

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access that start with 'DevinAI' or '@devin'.

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Folded into #39 (commit 55e941e on devin/1789702581-lilo-27b-256k-multinode) — the multi-node code it fixes hasn't shipped yet, so the fix belongs in that PR.