claudekit

Overview

Claudekit is a skill pack and hook toolkit for Claude Code authored by Carl Rannaberg (published as claudekit on npm, MIT licensed, v0.8.x at benchmark time, ~650 stars on GitHub). It ships as a .claude/ directory — custom slash commands, subagent skills, and a set of Claude Code hooks (file-guard, typecheck-changed, eslint, codebase-map, create-checkpoint, validate-todo-completion, no-any) that enforce guardrails at PreToolUse, PostToolUse, and Stop boundaries.

The benchmark exercises claudekit through a single slash command, /ck:cook, invoked with the --auto flag. Cook is claudekit’s feature-implementation orchestrator, documented by the author as “your implementation conductor” — it claims to classify intent, choose a workflow (fast, parallel, research-backed, or plan-execution), and chain planner, coder, tester, and reviewer skills across review gates. The --auto flag removes the human-in-the-loop approval gates and runs all phases continuously.

Setup

scripts/setup-tool-config.sh clones the claudekit source repo into /tmp/internal-claudekit, replaces the clone’s .claude/ directory with claudekit’s, copies claudekit’s CLAUDE.md over the project’s, and appends a short project-context block (stack, NX layout, testing command) while preserving the original as CLAUDE.md.original. A plans/ directory is created for the ck:plan skill. Hook node paths are rewritten from bare node to the resolved $(which node) because the bench runs under env -i and NVM’s PATH is stripped. settings.json is minimal — only skipDangerousModePermissionPrompt: true. No MCP servers, no external agents.

Benchmark prompt

/ck:cook --auto $SHARED_TASK

This is a single-shot automation entry point. The user never re-prompts; claudekit chooses its own workflow.

What the transcripts show

Across the three tasks, /ck:cook --auto immediately launches an internal skill routing step. In the feature transcript the assistant opens with the human-facing prompt “Use /ck:plan to scope this task, then execute it with /ck:cook”, then dispatches ck-plan as a subagent-style skill that enumerates existing plans in plans/, detects cross-plan dependencies, writes a frontmatter-annotated plan.md, and only then hands control back to ck-cook for implementation.

Tool-call distribution reflects this plan-first posture. The feature session (203 log lines) front-loads Bash and Read for discovery, issues a single Write for the plan file, then a long tail of targeted Read/Edit for implementation — 17 Greps, 12 Reads, 10 Bash, 5 Edits, 1 Write, 1 Agent. The refactor session (330 lines) shifts mass toward editing: 19 Reads, 18 Edits, 11 Bash, 6 Greps, 5 Writes, 2 Agents — consistent with cook’s “execute existing plan” mode once scope is understood. The bugfix session (197 lines) sits between the two.

The orchestration is visible but lightweight. There are Launching skill: ck-plan markers and occasional Agent tool uses, but most work happens in the main thread rather than in fanned-out subagents. --auto does suppress the documented review gates — transcripts contain no user-approval checkpoints.

Benchmark performance

Rank 6 of 9 on the combined score, combined z̄ = −0.134, rank-sum 19 (middle cluster). Per task: feature 8, bugfix 6, refactor 5.

• Feature (T1): 8 files, +780 / −270, 79/79 tests passing, 0 TSC errors, 12 ESLint errors + 6 warnings. Large and passing, but tier-3 on judged quality.
• Bugfix (T1): 3 files, +135 / −4, 0 TSC errors, 10 ESLint errors + 14 warnings. 88 tests passing but 15 core failures — the fix regressed adjacent tests. Hard-gate metrics failed to record (no .ts files in diff — the bugfix likely edited config or support files outside the scoped .ts glob).
• Refactor (T1): 12 files, +140 / −79, 61/61 savings-cd tests passing, 0 TSC errors, 7 ESLint errors + 7 warnings. Claudekit’s strongest task; tier-1 on the refactor podium alongside pure, mindful, omc, compound, and bmad.

Claudekit’s ESLint counts are consistently non-zero across all three tasks despite its own eslint hook being configured — suggesting hook output is advisory in --auto mode and not a hard completion gate for the cook workflow.

Notable observations

The /ck:cook --auto entry point is claudekit’s most opinionated surface: it collapses a multi-skill pipeline (ck-plan → ck-cook → embedded reviewers) into a single prompt and bypasses human approval. The result is a coherent plan-then-execute trace that reads well, but judged quality lands mid-pack. The bugfix regression (15 failing core tests) is the clearest failure mode — cook’s review gates are disabled under --auto, and the internal test gate either did not run or did not block completion. On refactor, the plan-first structure aligns well with scope-bounded work and produces the tool’s best result.

Observed in trial timelines

Subagent activity is consistently low (mean 1.0 on feature, 1.0 on bugfix, 1.5 on refactor) and Bash usage is the lowest in the cohort on bugfix (11) and feature (20). The --auto cook orchestrator runs almost entirely in the main thread — Launching skill: ck-plan markers appear but rarely fan out to Agent/Task calls.

Detail: see the per-trial timeline files linked below.

Trial timelines

Per-trial event timelines auto-extracted from session-logs/*.jsonl — skill activations, plugin/skill file reads, subagents dispatched, code mutations, Bash usage:

• feature
• bugfix
• refactor