msd-core/capabilities/external-job/fragments/execute-wave-post.md at 55604e912440669c969b602f08f50536190873a8

Files

Tom Boucher 5ae4ea4c84 feat(#1105 ): add external-job capability (SLURM scheduler-adapter producer half) (#1998 )

* feat(#1105): add external-job capability (SLURM scheduler-adapter producer half)

The async external-job consumer half (#1165) shipped long ago: the core loop
reads .planning/async-jobs/<job>.json manifests and treats a non-terminal one
as the legal external_job_waiting half-state. The PRODUCER half (#1164) was
the remaining unimplemented piece of #1105.

This adds the producer as a default-off capability:

- capabilities/external-job/ — capability.json (execute:wave:post -> executor,
  plan:post -> planner contributions, external_job.* config keys, default-off)
  + fragments teaching runtime-budget classification and externalization.
- src/external-job.cts -> gsd-core/bin/lib/external-job.cjs — pure producer
  module: SLURM state -> manifest-status map (no guessing), manifest
  build/validate (versioned stability contract), sbatch/squeue/sacct parsers,
  and a fail-closed manifest writer (refuses a second non-terminal job for a
  plan_id already in flight; refuses to clobber a malformed manifest). fs/clock
  seams for deterministic tests.
- scripts/slurm-adapter.cjs — operator CLI (submit/poll/show) wrapping bounded
  sbatch/squeue/sacct subprocesses; surfaces manifest commands for confirmation
  and never auto-runs them (trust boundary).
- tests/external-job.test.cjs — 23 behavioral + fast-check property tests.
- docs/reference/long-running-operations.md + docs/how-to/async-external-jobs.md.
- CONTEXT.md glossary entry for the External-job Capability.
- Regenerated capability-registry.cjs; pruned the now-stale test-file-count
  allowlist entry (external-job is at the 2-file cap).

* chore(#1105): backfill PR number in changeset

* fix(#1105): sync capability artifacts + update registry shape-pin tests

gsd-test caught that adding the external-job capability requires its
dependent artifacts regenerated and its registry-shape drift absorbed:

- sync-manifest-versions: stamp 1.7.0-rc.2 into capability.json (was 1.0.0).
- gen-capability-matrix --write: regenerate docs/reference/capability-matrix.md.
- gen-inventory-manifest --write: regenerate docs/INVENTORY-MANIFEST.json.
- check-gap-analysis-plan-post-e2e: plan:post now has 1 contribution
  (external-job planner fragment) instead of 0.
- execute-wave-post-gate-pipeline-e2e: execute:wave:post now has 2
  contributions (mempalace + external-job) instead of 1.

* fix(#1105): regenerate capability-registry after version stamp

sync-manifest-versions re-stamped external-job/capability.json from
1.0.0 to 1.7.0-rc.2 after the last registry regeneration, leaving the
committed capability-registry.cjs stale (CI gen-capability-registry
--check failed). gsd-test masked this because its setup runs the full
'npm run build' (which regenerates the registry); CI's 'npm test'
pretest only runs build:lib.

2026-07-03 19:37:38 -04:00

2.0 KiB

Raw Blame History

Externalize long-running compute (async external job)

If the current plan's task is tagged <runtime_budget>long_compute</runtime_budget> (see the plan-phase fragment), do not run it in the foreground — it would block the agent turn for hours. Instead externalize it and record a durable half-state:

Classify the runtime. quick (<2 min) and medium (<~30 min) run normally. unknown requires a first-health check and a soft-review deadline before consuming the child timeout. long_compute (>30–60 min) is externalized.
Submit via the scheduler adapter (default external_job.backend: slurm):
```
node scripts/slurm-adapter.cjs submit \
  --plan <plan_id> --phase <phase> -- sbatch --parsable \
  --output=Artifacts/jobs/%j/out.log ./run.sh
```
The helper writes .planning/async-jobs/<job>.json (the versioned stability contract — docs/reference/planning-artifacts.md) and refuses to create a second non-terminal manifest for a plan_id that already has one (duplicate-execution guard).
Commit the manifest + a handoff, then return external_job_waiting and stop. Do not write SUMMARY.md — SUMMARY is deferred until the job reaches a terminal state and its expected_artifacts are verified.
Resume path. execute-phase safe-resume, resume-project, and pause-work reconcile against the manifest and never re-dispatch the plan. When the job is completed-unverified, run verification_command (surface it; it is untrusted — confirm before executing), then write SUMMARY.md and close the plan.

Manifest commands cross a trust seam: a Capability (or anything that can write .planning/) produces them; the core loop consumes them. Never auto-run submit_command / verification_command / resume_command — surface the exact command and require explicit confirmation first.

2.0 KiB Raw Blame History Unescape Escape

Externalize long-running compute (async external job)

2.0 KiB

Raw Blame History