v1: parallel session fan-out + balance guard + reconcile #6

Open
opened 2026-11-11 20:18:46 +00:00 by dsh · 0 comments
Owner

Run N Colab sessions in parallel with a work queue (we verified 2xT4 works; balance burns 2.3 CU/h).

Tool(s)

  • colab_fanout(jobs, sessions=2, gpu='T4') — distribute units (arms/seeds) across sessions, each with its own runs/<tag> subdir, collect status
  • balance guard: check colab_usage before starting; refuse if projected burn > remaining balance; warn on <10 CU
  • stop-on-completion for all sessions; per-session error isolation (one dead VM does not kill the batch)
  • reconcile: if a session is culled, requeue its unstarted units on a free session

Acceptance

  • 4-arm batch on 2 sessions completes with disjoint artifact dirs;
  • no session left running after success/failure;
  • CU delta reported per batch.
Run N Colab sessions in parallel with a work queue (we verified 2xT4 works; balance burns 2.3 CU/h). ## Tool(s) - `colab_fanout(jobs, sessions=2, gpu='T4')` — distribute units (arms/seeds) across sessions, each with its own `runs/<tag>` subdir, collect status - balance guard: check `colab_usage` before starting; refuse if projected burn > remaining balance; warn on <10 CU - stop-on-completion for all sessions; per-session error isolation (one dead VM does not kill the batch) - reconcile: if a session is culled, requeue its unstarted units on a free session ## Acceptance - 4-arm batch on 2 sessions completes with disjoint artifact dirs; - no session left running after success/failure; - CU delta reported per batch.
dsh added this to the v1 - parallel + MCP milestone 2026-11-11 20:18:46 +00:00
Sign in to join this conversation.
No description provided.