How new engineers and agents work in this repo

Benchmark local agents live

Users run their own agents from local machines against ClawBench tasks. ClawBench scores the runs, publishes traces and leaderboards, and turns that evidence into shareable competitive content.

Improve agents on tasks

The roadmap tab should show what exists competitively now and what comes next: agent harness engineering, custom task onboarding, and task-specific improvement loops.

AI consulting surface

Consulting is a future track. Do not present it as shipped or guaranteed. It can be referenced as to be announced when owners decide the positioning.

Boundary

BenchFlow is the ACP benchmark runner. It creates task environments, launches the selected agent adapter, passes prompts, captures trajectories, runs verifiers, and writes artifacts.

ClawBench stays the product surface: public agents, run IDs, benchmark IDs, leaderboards, traces, redaction, visibility, and ticket/PR discipline.

Run contract

Use execution_mode: benchflow_acp for BenchFlow-backed runs. Keep public ClawBench agent names separate from BenchFlow adapter names such as claude-agent-acp, codex-acp, opencode, and openclaw.

Import result.json rewards, verifier errors, timing, and trajectory/acp_trajectory.jsonl into ClawBench outputs, events, traces, and score fields.

Install and run locally

npm install
uv sync

docker run --name clawbench-pg \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_DB=postgres \
  -p 54322:5432 -d postgres:16

cat apps/backend/sql/001_initial_schema.sql \
  | docker exec -i clawbench-pg psql -U postgres -d postgres

Run app surfaces

DATABASE_URL=postgresql://postgres:postgres@127.0.0.1:54322/postgres \
FASTAPI_BASE_URL=http://127.0.0.1:8080 \
CLAWBENCH_PUBLIC_APP_ORIGIN=http://localhost:3000 \
CLAWBENCH_ADMIN_TOKEN=replace-me \
uv run uvicorn backend.main:app --host 0.0.0.0 --port 8080 --reload

VITE_API_BASE_URL=http://127.0.0.1:8080/api/v1 npm run dev

Shared working agreement

• Start from Linear tickets and use PRs for implementation.
• Respect benchmark catalog control and public run identity.
• Keep architecture changes inside router, service, repository, worker, and migration boundaries.
• Verify before claiming completion.

Behavioral guardrails

• Think before coding and surface uncertainty.
• Prefer minimum code and no speculative abstractions.
• Make surgical changes and leave unrelated code alone.
• Transform work into verifiable goals and checks.

How to adhere

Use docs/repo-system-map.html for final-state architecture, apps/backend/ARCHITECTURE.md for backend boundaries, and the current Linear ticket for local scope. If they conflict, stop and ask the owner to resolve it.

How to adhere

Use Supabase migrations for database changes, GitHub Actions for CI/CD, Cloud Run for backend/API deployment, Vercel for frontend deployment, and Cloudflare Pages only for static docs/reports.

Commands

• npm test: Vitest frontend and TypeScript contract tests.
• npm run build: sitemap generation plus Vite production build.
• npx tsc --noEmit: TypeScript type check when TS surfaces change.
• uv run pytest: Python service/API/repository tests.
• python -m py_compile $(find apps/backend scripts -name '*.py' -print): Python syntax check.

CI coverage

.github/workflows/deploy.yml runs npm test, npm run build, selected Python onboarding/agent tests, Python compile checks, Supabase migration validation, and production deploy jobs on main.

Find what needs doing

• Start in Linear Backlog and In Progress queues.
• Use parent/child relationships around program tickets such as MCP custom-agent work.
• Read the Roadmap tab for product direction and visible gaps.
• Read docs/repo-system-map.html before changing architecture.
• Search local plans with rg --files docs/superpowers/plans.

Find what already happened

• Check Linear Done tickets for closure comments and validation evidence.
• Search merged PRs and branches by KAN ID.
• Use git log --oneline --grep KAN- when commit messages include ticket IDs.
• Check GitHub Actions for post-merge deploy and migration results.
• Use trace and leaderboard tests to understand what behavior is already protected.