No description
  • JavaScript 100%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
dsh 4c631a3903
All checks were successful
CI / check (push) Successful in 44s
Release notes / release (push) Successful in 36s
Publish package / publish (push) Successful in 1m8s
Publish to npmjs / publish (push) Successful in 1m11s
chore(release): v0.1.7
2026-11-12 18:25:14 +00:00
.forgejo/workflows
docs fix(drive): Ask Next/Cancel before sending the consent Enter 2026-11-12 16:49:55 +00:00
lib fix(drive): show one short Colab Drive Auth link 2026-11-12 18:25:14 +00:00
locale
scripts
test fix(drive): show one short Colab Drive Auth link 2026-11-12 18:25:14 +00:00
.gitignore
AGENTS.md
CHANGELOG.md chore(release): v0.1.7 2026-11-12 18:25:14 +00:00
cordis.patch.yml
icon.svg
LICENSE
package.json chore(release): v0.1.7 2026-11-12 18:25:14 +00:00
README.md fix(drive): Ask Next/Cancel before sending the consent Enter 2026-11-12 16:49:55 +00:00

dsh-colab

Agent-native Google Colab tools for DeepSeek Harness: session lifecycle, the Drive consent handoff, attached GPU jobs with artifact verification, quota awareness — plus an optional adapter for the beta Colab API MCP server.

Status: 0.1.5 — v0 CLI wrapper implemented and tested offline; the MCP adapter is opt-in and beta. Published to the pvlb.dev Forgejo registry from a signed v0.1.2 tag and staged on npmjs (approve the stage with 2FA to publish there; see Releases). Nothing in this plugin ever touches a Colab session until one of its tools is called.

Why

Multi-hour GPU runs on Colab work, but the workflow lives in shell history: create a session, mount Drive (drivemount needs a browser click and Enter), upload a bundle, keep a long exec attached so the VM is not idle-culled, copy evidence to Drive after every unit of work, stop the VM so quota stops burning. Every project re-implements it, and a run that failed to back up its metrics can still look successful.

dsh-colab turns that workflow into typed DSH tools. A project supplies a bundle, a setup script, a runner and a tag.

Requirements

Host a machine where the agent runs and the operator has a colab login (Linux/macOS)
DSH 0.2.2-rc.1 (peer ranges declared in package.json)
CLI google-colab-cli 0.7.5 – 0.8.x (uv tool install google-colab-cli)
Auth done once outside DSH: run colab new in a terminal and finish the OAuth flow
GPU none on the host; the accelerator is the remote VM

The plugin never stores credentials. It uses the CLI's own ~/.config/colab-cli/ state and the DSH credential store only for the optional MCP token.

Install

# published package (Forgejo registry; npmjs once the stage is approved)
dsh plugin --profile dev add --registry https://git.pvlb.dev/api/packages/dsh-plugins/npm/ dsh-colab

# or the repository directly
dsh plugin --profile dev add ssh://git@git.pvlb.dev/dsh-plugins/dsh-colab.git

# or a local checkout (development)
dsh plugin --profile dev add ~/Projects/dsh-colab
node scripts/dev-link-dsh.mjs        # linked checkouts only: makes peer imports resolvable

Restart the profile after installing, then call colab_plugin_info in a fresh session: it reports the CLI path/version, the Drive paths, and whether the MCP adapter is enabled.

The bundle patch mounts one Host plugin row with an empty config, so every option below is optional:

plugins:
  colab:
    project: hijack
    gpu: T4
    drive_root: /content/drive/MyDrive

Tools

Read-only tools are safe at any time. Mutating tools act on the operator's real Colab account and burn compute units while a session exists.

tool kind what it does
colab_plugin_info read-only CLI path/version, Drive paths, MCP status
colab_sessions read-only live sessions with hardware/shape/variant
colab_session_status read-only state, last execution; all sessions when omitted
colab_usage read-only compute-unit balance, hourly rate, active assignments
colab_log read-only structured session history (colab log -o …jsonl)
colab_ls read-only list a remote path
colab_session_new mutating create a runtime and wait for IDLE; refuses an existing name unless reuse
colab_session_stop mutating stop one explicitly named session (idempotent; no wildcard)
colab_session_restart mutating restart the session kernel
colab_drive_auth mutating drive drivemount through the consent: asks a Next/Cancel question with the clickable URL, sends Enter only after Next, detects the mount
colab_upload mutating upload a file to the VM/Drive with retry + verify; overwrite guard
colab_download mutating download a remote file with retry + verify
colab_exec mutating run a local script attached, with a wall-clock timeout
colab_run mutating session → Drive → bundle → setup → runner → artifact invariant → stop
colab_mcp_status read-only Colab API MCP connection, auth mode, registered tools
colab_mcp_refresh read-only re-list MCP tools and register new ones

The Colab CLI tools return a uniform envelope:

{ "ok": true, "session": "hijack", "data": { } }
{ "ok": false, "session": "hijack", "error": { "code": "artifact_missing", "message": "…", "hint": "…", "raw_tail": "…" } }

Failed long runs also carry a partial data report (what ran, what was observed) so a failure can be diagnosed without the session. The two MCP diagnostics (colab_mcp_status, colab_mcp_refresh) return flat status objects instead, matching the MCP adapter convention; MCP-discovered tools return the upstream server's content.

Safety and cost model

  1. The VM is disposable; Drive is the truth. colab_run verifies that the configured heartbeat (runs/<tag>/manifest.json by default) exists, is non-empty, and advanced during the run. If not, it fails with artifact_missing — a run that did not back up its evidence is never reported as successful.
  2. Always stop. colab_run stops the session on success, failure, timeout and exception (stop_on_finish, default true). keepSessionsOnFailure is a diagnostics-only escape hatch; it keeps burning quota.
  3. Stops are explicit. colab_session_stop requires a session name and has no all/wildcard argument, so one agent cannot kill another agent's run by accident. Already-absent sessions are a successful no-op.
  4. One human action. colab_drive_auth surfaces the consent URL in its result (DSH renders it as a link), waits, presses Enter, and detects /content/drive from inside the VM. The plugin never auto-approves OAuth and never accepts tokens as arguments.
  5. No secret logging. OAuth codes, bearer tokens, CLI proxy tokens and the session state file never enter a result tail, a log line or a commit. The short-lived consent URL is the one intentional exception, and only in the tool result.
  6. No silent guesses. Text parsing is pinned to the validated CLI range. An unknown format fails with cli_parse plus a redacted tail instead of a made-up value.
  7. Long runs stay attached. colab exec --timeout is an idle timeout, so the plugin enforces its own wall-clock ceiling (maxExecSeconds) and can always stop the VM afterwards.

Configuration

All fields have defaults; override only what differs.

field default meaning
cliBin colab CLI executable (absolute path or wrapper)
cliVersionRange >=0.7.5 <0.9.0 advisory range the parser was validated against
gpu / tpu / highMem T4 / `` / false default accelerator request
sessionPrefix colab prefix for generated session names
project colab Drive root: <driveRoot>/<project>/runs/<tag>/
driveRoot /content/drive/MyDrive Drive mount root inside the VM
runsDir runs artifact directory under the project
commandTimeoutMs 180000 wall clock for ordinary commands (uploads, downloads, session calls)
listTimeoutMs 60000 wall clock for one colab ls probe; the kernel serializes work, so a listing can queue behind a running cell
execIdleTimeoutSec 900 passed to colab exec --timeout (idle guard)
maxExecSeconds 28800 plugin wall-clock ceiling for one exec
maxOutputBytes 262144 bounded stdout/stderr capture
defaultStopOnFinish true stop the session when colab_run ends
driveOpenBrowser true best-effort launch of the Drive consent URL; the URL is always returned in the result
keepSessionsOnFailure false diagnostics only: leave a failed session running
mcpEnabled false connect to the Colab API MCP server
mcpUrl https://colaboratory.googleapis.com/mcp MCP endpoint (v1beta)
mcpServerName colab namespace of MCP tool names
mcpAuthMode none none, bearer, or command
mcpTokenEnv COLAB_MCP_ACCESS_TOKEN DSH credential reference (never the value)
mcpTokenCommand `` executable whose stdout is an access token
mcpTools / mcpExcludeTools [] / [] allowlist patterns (*) / exclusions
mcpToolCallTimeoutMs 60000 per MCP tool call
mcpFailOnStartupError false fail activation when the MCP server is unreachable

MCP adapter (beta)

Google ships a Colab API MCP server with structured tools: list_runtime_specs, create_runtime, get_runtime, list_runtimes, delete_runtime, create_session, list_sessions, delete_session, execute_code, get_subscription. execute_code streams output and can be reattached with a stable executionId for up to 600 s of stream; it has no file upload/download and no Drive mount, so the CLI remains the right tool for persistence.

The adapter registers whatever the server advertises as mcp__colab__<tool> at activation and again on colab_mcp_refresh, so new server-side tools appear without a plugin release.

Auth is per-account Google OAuth. Setup, once:

  1. enable MCP servers and authenticate for the account (see Google's Cloud MCP authentication);
  2. give the plugin a short-lived access token, either as a DSH credential (mcpAuthMode: bearer + mcpTokenEnv) or through a command (mcpAuthMode: command, e.g. gcloud auth print-access-token).
plugins:
  colab:
    mcpEnabled: true
    mcpAuthMode: command
    mcpTokenCommand: gcloud auth print-access-token
    mcpTools: ['execute_code', 'list_*', 'get_*']

colab_mcp_status reports connection, credential presence (never the value) and registered tools. See docs/mcp-spike.md for the tool matrix, the open questions (cross-visibility with the CLI, reattach behavior) and the hybrid recommendation.

Errors

code meaning / action
cli_missing colab not found; install it or set cliBin
cli_auth the CLI login expired; run colab new interactively once
cli_timeout / exec_timeout CLI (or the attached exec) exceeded its budget
cli_parse output did not match the validated CLI format; tail included
session_exists / session_not_found caller error; colab_sessions shows the truth
session_required multiple sessions and no explicit session argument
session_lost the VM disappeared or was pruned mid-call
no_gpu / too_many_sessions accelerator entitlement / capacity (Colab refused)
auth_pending the consent was not completed in time; URL included, retry
mount_missing /content/drive absent after auth
transfer_failed upload/download failed after retries
artifact_missing the runner did not back up evidence; do not trust the run
mcp_auth / mcp_disabled / mcp_unavailable adapter state
aborted the tool call was cancelled

Releases

The repository ships the house four-workflow layout in .forgejo/workflows/ (details and recovery notes: docs/releasing.md):

workflow trigger what it does
ci.yml push to master, manual npm run check + npm pack --dry-run
publish.yml tag v*, manual = auth check + dry run publishes to the pvlb.dev Forgejo npm registry with an authorized-integration OIDC token (AUDIENCE variable; optional NPM_TOKEN fallback)
publish-npmjs.yml tag v*, manual = checks only stages the version on npmjs (NPMJS_TOKEN, stage-only); a maintainer approves with 2FA
release.yml tag v*, manual = notes preview creates the Forgejo release from CHANGELOG.md and attaches the tarball
# one release, all artifacts; the tag must equal v<package.json version>
git tag -s v0.1.0 -m "dsh-colab v0.1.0"
git push origin v0.1.0

Every workflow is idempotent: an existing registry version, npmjs version or release page is skipped, never overwritten. No workflow runs on pull_request — the runner exposes the host Docker socket and the repository is public.

Development

node scripts/dev-link-dsh.mjs   # symlink DSH peers into ./node_modules (linked checkouts)
npm test                        # offline unit tests; fake colab CLI only
npm run check                   # syntax check + full suite
npm run pack:check              # inspect the published file list

The test suite never calls a real colab binary: test/fake-colab.mjs reproduces the CLI's documented output and a small remote filesystem, and every test drives it through test/helpers/harness.mjs. There is deliberately no live test in CI.

An opt-in live smoke exists for a machine with the real CLI and a spare ~1 CU:

DSH_COLAB_LIVE_SMOKE=1 node scripts/smoke-live.mjs --confirm-live

It creates a uniquely named T4 session, mounts Drive, runs one tiny script, verifies the artifact, and stops the session — read the script before running it and only do so when you own the account and no other run is active.

Non-goals

  • Notebook/browser automation.
  • Consumer-Colab inference serving or tunnels (ToS; see issue #9).
  • Replacing project experiment runners: colab_run orchestrates yours.

Docs

License

MIT — see LICENSE.