maga2010org/hackathon-run

Ship the demo, not the dream.

A decision-making and execution system for hackathon teams operating under time pressure. Fifteen skills, one workflow: clarify, prize-target, scope, time-box, build, verify, demo, judge, ship, recover, pivot, retro, decide-log.

A hackathon is not a coding problem. It is a time-pressure, decision-making, execution problem.

• 23 hours in, you have 8 unfinished features.
• The demo crashes on stage and you have 60 seconds to recover.
• A judge asks "what's novel here?" and you have no answer.
• Your README references API keys that you cannot ship.
• Your teammate has been debugging the wrong thing for 4 hours.

Hackathon Run does not help you write code faster. It helps you make the right cut, at the right time, every time.

Fifteen skills, mapped to the hackathon lifecycle:

idea-clarify (pre) pivot (mid-build redirect) │ │ ▼ ▼ scope-knife ─► time-box ─► fast-verify ─► demo-coach ─► judge-sim ─► ship-pack │ │ │ │ │ │ └──────────────┴────────────┴──────────────┴─────────────┴────────────┘ │ │ │ │ │ │ stack-picker (cold-start) retro (post-event) │ │ │ │ team-roster (build start) recovery-runbook (anytime) │ │ demo-rehearsal (final 2h)

Each skill is independently invokable. You can run any of them at any time without running the others.

Hackathon Run works best when an agent treats it as a harness, not as a menu of one-shot prompts. Four roles keep long-running work moving without letting the same agent both build and approve its own output: an initializer sets up the first session, a planner writes the default-FAIL contract, a generator builds one feature per sprint, and an evaluator verifies it from a fresh context.

The runtime follows the production agent-loop pattern used by ChatGPT and the OpenAI Agents SDK: context is assembled from sessions, every input and output passes a guardrail, tools return observable results, the loop is bounded by budget, and every meaningful step is traced. It also follows Anthropic's long-running harness pattern: the first session initializes the environment, every later session reads PROGRESS.md + git log, and the operator can stop or steer the loop from the outside.

flowchart LR
  User(["User / trigger"]) --> InGuard{"Input guardrail\npolicy + budget + schema"}
  InGuard -->|"reject"| Block(["Blocked\nrefuse + explain"])
  InGuard -->|"accept"| Context["Context assembly\nsession + plan + skill"]
  Context --> Loop{"Agent loop\nmax_turns + budget"}
  Loop -->|"next turn"| Reason["Reason\nchoose action"]
  Reason --> Tools["Tool invocation\nskills / scripts / MCP / shell"]
  Tools --> Observe["Observe\nstdout / files / tests / evidence"]
  Observe -->|"loop"| Loop
  Loop -->|"final"| OutGuard{"Output guardrail\nJSON Schema + evidence"}
  OutGuard -->|"reject"| Loop
  OutGuard -->|"accept"| Output(["Final output\nstate + evidence"])
  Context -. "read / write" .-> Session[("Session\nhandoff + memory")]
  Loop -. "trace" .-> Trace[("Trace\nevents / spans")]

The production agent loop is organized into six layers: interface, context, agent loop, hands, durable state, and operator control. Every layer writes to or reads from the durable state store so a fresh context window can resume without the previous conversation.

flowchart TB
  classDef state fill:#fff7ed,stroke:#ea580c,color:#7c2d12;
  classDef gate fill:#eff6ff,stroke:#2563eb,color:#1e3a8a;
  classDef trace fill:#f0fdf4,stroke:#16a34a,color:#14532d;

  subgraph Interface["Interface Layer"]
    User(["User / trigger"]) --> InGuard{"Input guardrail\npolicy / budget / schema"}
    InGuard -->|"reject"| Reject(["Blocked\nrefuse + explain"])
    InGuard -->|"accept"| ContextAssembly["Context Assembly\nplan + session + skill + progress"]
  end

  subgraph Context["Context Layer"]
    Session[("session.json\nhandoff")]
    Progress[("PROGRESS.md\nagent log")]
    Git[("git log\ncommit history")]
    ContextAssembly -. "reads" .-> Session
    ContextAssembly -. "reads" .-> Progress
    ContextAssembly -. "reads" .-> Git
  end

  subgraph Loop["Agent Loop Layer"]
    ContextAssembly --> LoopGate{"Loop\nbudget / max_iterations"}
    LoopGate -->|"turn"| Reason["Reason\nchoose action"]
    Reason --> ToolGate["Tool invocation"]
    ToolGate --> Observe["Observe\nstdout / files / tests / evidence"]
    Observe -->|"iterate"| LoopGate
    LoopGate -->|"final"| OutGuard{"Output guardrail\nJSON Schema / evidence"}
    OutGuard -->|"reject"| LoopGate
  end

  subgraph Hands["Hands Layer"]
    Skills["skills / scripts"]
    MCP["MCP tools"]
    Browser["browser automation"]
    Shell["shell / sandbox"]
    ToolGate --> Skills
    ToolGate --> MCP
    ToolGate --> Browser
    ToolGate --> Shell
  end

  subgraph State["Durable State Layer"]
    Plan[("plan.json\nP0/P1/P2 + default-FAIL")]
    Sprint[("sprint.json\ncriteria + budget + rubric")]
    Eval[("eval.json\nscores + strategy")]
    Trace[("events.jsonl\nappend-only")]
    OutGuard -->|"accept"| Plan
    LoopGate -. "write" .-> Sprint
    LoopGate -. "write" .-> Eval
    LoopGate -. "trace" .-> Trace
  end

  subgraph Operator["Operator Layer"]
    Stop[("AGENT_STOP\nkill switch")]
    Steer[("STEER.md\none-shot redirect")]
    Stop -. "halts" .-> LoopGate
    Steer -. "surfaced once" .-> ContextAssembly
  end

  class Session,Progress,Git,Plan,Sprint,Eval,Trace state;
  class InGuard,OutGuard,LoopGate,Stop gate;

The same architecture implemented with real commands, role prompts, and state artifacts:

flowchart TB
  classDef contract fill:#fff7ed,stroke:#ea580c,color:#1f2937;
  classDef gate fill:#eff6ff,stroke:#2563eb,color:#1f2937;
  classDef trace fill:#f0fdf4,stroke:#16a34a,color:#1f2937;
  classDef failure fill:#fef2f2,stroke:#dc2626,color:#1f2937;

  subgraph First["0. Initializer / First Session"]
    direction TB
    Brief(["User brief"]) --> InitCmd["hackathon init"]
    InitCmd --> ScopeCmd["hackathon run scope-knife --apply"]
    ScopeCmd --> Plan[("plan.json\nP0/P1/P2 + default-FAIL")]
    ScopeCmd --> Session[("session.json\nhandoff")]
    InitCmd --> Progress[("PROGRESS.md\nagent-maintained")]
    Progress --> Smoke["start app + smoke test\nverify demo path"]
    Smoke --> FirstCommit["git commit\ninitial setup"]
  end

  subgraph Orchestration["1. Orchestration / Handoff"]
    direction TB
    FirstCommit --> Resume["hackathon resume\npwd -> git log -> PROGRESS.md -> smoke"]
    Resume --> Route{"Next unpassed\nP0/P1/P2 KEEP feature?"}
    Route -->|"yes"| Contract["hackathon sprint new --feature X"]
    Contract --> Approve["hackathon sprint approve"]
    Approve --> Sprint[("sprint.json\ncriteria + budget + rubric")]
    Sprint --> Build
    Route -->|"no"| Pipeline["Delivery pipeline\nfast-verify -> demo-coach -> judge-sim -> ship-pack"]
  end

  subgraph Generator["2. Generator (generator.md)"]
    direction TB
    Build["Read contract + session\nbuild one feature"] --> Self["Self-verify\nlint / tests / smoke"]
    Self --> Commit["git commit + update session"]
    Commit --> Checkpoint["hackathon checkpoint\nappend PROGRESS.md"]
  end

  subgraph Evaluator["3. Evaluator (evaluator.md)"]
    direction TB
    Checkpoint --> Review["hackathon sprint review"]
    Review --> Eval[("eval.json\ncriteria default false")]
    Eval --> Run["Run app / tests / browser / commands"]
    Run --> Evidence["Collect machine-checkable evidence"]
    Evidence --> Rubric{"Score rubric 0-5\nweight + hard threshold"}
    Rubric --> Verdict{"Every criterion\nmeets threshold?"}
    Verdict -->|"no"| Feedback["Write feedback + strategy\nto session.json"]
    Feedback --> Resume
    Verdict -->|"yes"| Accept["hackathon sprint accept"]
    Accept --> Plan
  end

  subgraph Safety["4. Guardrails, Operator Controls & Observability"]
    direction TB
    Budget{"Budget gate\nminutes / max-iterations"} -->|"exhausted"| Blocked["sprint blocked\nverdict = blocked"]
    Schema{"JSON Schema\n+ default-FAIL"} -->|"invalid"| Blocked
    Stop{"AGENT_STOP?"} -->|"exists"| Blocked
    Steer[("STEER.md")] -->|"surfaced once"| Resume
    Blocked --> Session
    Trace[("events.jsonl\nappend-only")] --> Observe["hackathon trace / replay / report"]
    Checkpoint -. "trace" .-> Trace
    Review -. "trace" .-> Trace
    Accept -. "trace" .-> Trace
    Pipeline -. "trace" .-> Trace
  end

  Pipeline --> Ship["Ready to submit"]
  class Plan,Session,Sprint,Progress,Eval contract;
  class Budget,Schema,Stop gate;
  class Trace trace;
  class Blocked,Feedback failure;

Each known long-running-agent failure mode is stopped by a specific gate:

flowchart LR
  classDef fail fill:#fef2f2,stroke:#dc2626,color:#1f2937;
  classDef gate fill:#eff6ff,stroke:#2563eb,color:#1f2937;

  FM1["Failure: agent tries to one-shot\nthe whole app"] --> G1{"Gate:\none feature per sprint\n+ default-FAIL"}
  FM2["Failure: agent declares victory\nbefore the demo works"] --> G2{"Gate:\nfresh-context evaluator\n+ machine-checkable evidence"}
  FM3["Failure: next session guesses\nwhat the previous one did"] --> G3{"Gate:\nPROGRESS.md + git log\n+ smoke before building"}
  FM4["Failure: unit tests pass but\nuser-visible flow is broken"] --> G4{"Gate:\nbrowser / command evidence\n+ sprint accept"}
  G1 --> Loop(["Next unpassed KEEP feature"])
  G2 --> Loop
  G3 --> Loop
  G4 --> Loop
  class FM1,FM2,FM3,FM4 fail;
  class G1,G2,G3,G4 gate;

The same loop at command level, including the failed-iteration feedback path:

sequenceDiagram
  autonumber
  participant I as Initializer
  participant O as Orchestrator (CLI)
  participant G as Generator
  participant E as Evaluator
  participant S as State Store
  participant T as Trace
  participant Op as Operator

  I->>O: hackathon init
  O->>S: seed session.json + PROGRESS.md
  I->>O: hackathon run scope-knife --apply
  O->>S: write plan.json (P0/P1/P2, default-FAIL)
  I->>I: start app + smoke test
  I->>O: git commit initial setup
  O->>G: hackathon resume
  G->>S: read PROGRESS.md + session.json + git log
  G->>O: hackathon sprint new + approve
  O->>S: write sprint.json (criteria + budget)
  G->>S: commit code + update session.json
  G->>O: hackathon checkpoint --summary "..."
  O->>S: append PROGRESS.md
  G->>E: hackathon sprint review
  E->>S: write eval.json (criteria default false)
  E->>E: run app / tests / browser / commands
  E->>E: score rubric 0-5 + set strategy

  alt all criteria pass
    E->>O: hackathon sprint accept
    O->>S: flip plan.json passes=true + evidence
  else criteria fail
    E->>S: write feedback + strategy to session.json
    E-->>G: feedback for the next iteration
    G->>G: rebuild against the same contract
  end

  Op->>O: hackathon guard steer / guard stop
  O->>S: write STEER.md / AGENT_STOP
  G->>O: hackathon resume
  O-->>G: surface steer once / refuse when stopped

  G-->>T: append event to events.jsonl
  E-->>T: append verdict to events.jsonl
  O-->>T: append state transition to events.jsonl

The harness runtime turns those roles into durable artifacts:

• plan.json is a default-FAIL contract. Every KEEP feature starts with passes: false; only evidence-backed evaluation can flip it.
• session.json is the handoff. A fresh agent reads it to resume work without replaying the previous conversation.
• sprint.json is the definition of done. The generator and evaluator agree on criteria before code is written.
• eval.json is the evaluator verdict. Failed criteria return actionable feedback to the generator for the next iteration.
• events.jsonl is the append-only trace. hackathon trace, replay, and report reconstruct what actually happened.

# One complete agent loop
hackathon init
hackathon run scope-knife --demo-goal "sign up + save note" --time-remaining 240 --apply
hackathon resume
hackathon sprint new --feature Auth
hackathon sprint approve

# Generator builds Auth against the contract, then:
hackathon sprint review

# Evaluator fills .hackathon/state/eval.json with evidence and feedback, then:
hackathon sprint accept
hackathon trace
hackathon flow --execute --demo-goal "sign up + save note" --time-remaining 240

For automated verification, add a command to each demo_path step in .hackathon/state/plan.json, then run hackathon verify run. It executes every step in order, checks the expected output, stops at the first failure with a diagnosis, writes the workspace digest to verify.json, and syncs plan.features[].passes. Steps without commands are recorded as skip; they do not count as a verified demo, and the pipeline will not report ready to ship. flow --execute runs the same executor as stage 2 of the pipeline.

State files that already hold a complete artifact are protected: hackathon run <skill> --apply refuses to replace them with an empty skeleton. Pass --force to overwrite deliberately — verified evidence is never silently clobbered.

sprint accept applies the evaluator verdict: a passing eval flips the feature to passes: true and records evidence; a failing eval writes feedback back to session.json for the next generator iteration.

scope-knife assigns demo_path[].feature to the core product steps. Fast-verify synchronizes those steps back to plan.features[].passes: every owned executable step must pass, while a failure or skip resets the feature to false. Auto-generated evidence is tagged source: fast-verify; evaluator and manual evidence are preserved.

To keep a fresh session small without losing raw history, run:

hackathon checkpoint --summary "what changed" --compress

This rewrites .hackathon/SESSION.md to 150 lines or fewer. The full PROGRESS.md log and append-only events.jsonl trace remain intact.

Every new trace event carries a sequence number and SHA-256 hash chain. verify.json also records a workspace digest, so source edits mark prior evidence stale instead of leaving a false passes: true.

hackathon trace --verify

hackathon mcp exposes the same 25 tools over JSON-RPC 2.0, but MCP calls no longer scrape CLI stdout. CLI commands return typed result payloads, and the MCP layer validates every tool argument against its JSON Schema before dispatch.

Tool results include both backward-compatible text JSON and MCP structuredContent. Expected command failures set isError: true instead of becoming transport errors. apply_skill_advice validates the target state schema and writes through the same cross-process lock and atomic rename path as the CLI.

See MCP protocol for the response shape and failure semantics.

Cost and time are first-class gates. Set them when creating a sprint:

hackathon sprint budget --minutes 45 --max-iterations 3

Trace is on by default for initialized projects. Disable it with HACKATHON_TRACE=0 when you do not want runtime event noise.

Run an A/B measurement before adding more harness machinery:

npm run ab:harness -- \
  --solo-command "hackathon run scope-knife" \
  --harness-command "hackathon flow --execute"

Requires Node.js 20.9+ and Python 3.11+ for the script-backed skills. The CLI resolves Python in this order: PYTHON, python3, python, then the Windows py -3 launcher. hackathon doctor, hackathon flow, and the evaluation lab all use the same resolver, so an explicit PYTHON override behaves consistently on Windows, Git Bash, WSL, and CI.

# Option A -- one-shot via npx (no install needed)

npx @hackathon-run/hackathon-run init

# Option B -- install globally, then use hackathon as the CLI command

npm install -g @hackathon-run/hackathon-run
hackathon init

# Option C -- from source

git clone https://github.com/MAGA2010/hackathon-run
cd hackathon-run
npm install
npm run build
npm link

# Inside any hackathon project

cd my-hackathon-project
hackathon init # creates .hackathon/ in your repo
hackathon run scope-knife # forces a KEEP/CUT/DEFER decision
hackathon run fast-verify # verifies the demo path
hackathon run demo-coach # drafts the pitch
hackathon run judge-sim # self-reviews before submitting
hackathon run ship-pack # packages and checks for leaks
hackathon status --watch --interval 30 # live time and rehearsal-risk pulse
hackathon resume # handoff brief for a fresh agent or new session
hackathon sprint new --feature Auth # create a default-FAIL contract
hackathon sprint approve # lock the contract before building
hackathon sprint review # emit the evaluator handoff
# Evaluator fills eval.json, then:
hackathon sprint accept # apply the verdict back to plan/session
hackathon trace # inspect the runtime event log

# Chained run: follows Format v2 dependencies automatically
hackathon run demo-rehearsal --chain # scope-knife -> fast-verify -> demo-coach -> demo-rehearsal

State is saved to .hackathon/state/ and is never required by the next step. hackathon init seeds placeholder files so the schema and paths exist, but status, resume, and flow do not treat that scaffold as completed work. The shared lifecycle snapshot checks artifact content instead: a placeholder plan keeps the pipeline at empty, a passing verification moves it to demoing, and a clean ship audit is required before the pipeline reports complete.

When time-box.json, rehearsal.json, or verify.json evidence exists, hackathon status also prints a Pulse section with remaining time, stage burn, rehearsal risk, and a suggested recovery severity. Use hackathon status --watch during the final countdown.

The clock telemetry feeds a calibration loop: when retro runs, time-box.json is distilled into a calibration record inside retro.json — planned vs observed minutes per stage, overrun ratios, alarms fired, and the final burn rate. Across events, those records become the dataset for adjusting the stage-budget table instead of guessing.

After install, the CLI command is hackathon (not hackathon-run). The package is @hackathon-run/hackathon-run; the binary is hackathon.

For CI, run hackathon skills lint to validate every bundled skill and hackathon skills audit to statically review skill security. Pin the team's skill versions with hackathon skills pin --all. Routing uses a local BM25 hybrid by default, with an optional semantic matcher behind HACKATHON_EMBED_BACKEND.

hackathon skills audit --json
hackathon skills audit --risk-summary --json
hackathon skills audit --strict --policy .hackathon/skills-policy.json
hackathon skills audit --sarif > skills-audit.sarif
npm run test:routing

The audit compares declared capabilities (fs_read, fs_write, net, exec, env, mcp) against scripts, supports deny-policies, and emits SARIF 2.1.0 for code-scanning integrations. A bundled skill can acknowledge a reviewed finding with audit_waive: ['shell.child-process', ...] frontmatter — medium and below only; critical and high findings always surface.

Skill routing is bilingual: English and Chinese queries both hit the same BM25 + trigger-phrase pipeline (han bigram indexing, CJK substring phrase match). The 109-case golden set covers 44 Chinese / mixed utterances and Chinese adversarial negatives, gating precision, recall, and false-positive rate in CI.

judge-sim sends protocol v2 requests to HACKATHON_JUDGE_BACKEND, requiring per-dimension rationale, evidence, and confidence while still accepting older v1 responses. Calibrate a backend against a golden set with:

hackathon judge-calibrate \
  --backend https://judge.example.test \
  --golden tests/fixtures/judge-golden.json \
  --max-mae 1

The deterministic evaluation gate remains offline. A manual or scheduled model-backed evaluation can run Codex, Claude Code, or any command template:

npm run test:skill-eval

node skill-eval-lab/harness.mjs \
  --runner command \
  --runner-preset codex \
  --runs 3 \
  --ab \
  --control-command "your-baseline-agent --prompt-file {prompt_file}" \
  --min-grade A \
  --fail-on-p1

CI writes routing-report.json plus skill-eval-lab/results/report.md, final.json, and runs/aggregate.json, then uploads them as the quality-reports artifact. See the evaluation lab for the scoring model, scenario layout, and cross-platform commands.

Release tags run the reusable CI workflow, verify that the v* tag exactly matches package.json, run npm pack --dry-run, and only then publish to npm. The local npm run release path applies the same routing, evaluation, lint, format, build, and package preflight gates before creating the tag.

Third-party skills can ship a full manifest (license, author, homepage, repository, compatibility) that hackathon skills search --json and the find_skills MCP tool surface.

1. Each skill is independently usable. No forced flow. Run any skill at 2am without reading the docs first.
2. State lives in the filesystem (.hackathon/state/*.json), readable, never blocking.
3. Trigger phrases in every description so the agent can match intent without ambiguity.
4. Body = execution logic only. No backstory, no changelogs inside skill files.
5. v1 ships a small set of skills done well, not 100 stubs.
6. Acceptance criteria live in the skill file and are wired to shell tests.

Full docs in docs/index.md.

• Getting Started
• 36-Hour Walkthrough
• Skill Reference
• Architecture
• Contributing

We welcome skill proposals, bug fixes, and docs improvements. See CONTRIBUTING.md.

The skill template is the source of truth for adding new skills.

MIT — © 2025-2026 MAGA2010