Session audit — per-tool behavioral metrics

Mined from results/<tool>/t<N>/session-logs/*.jsonl (feature) and results/{bugfix,refactor}/<tool>/t<N>/session-logs/*.jsonl (other tasks), including subagents/*.jsonl. Numbers are the mean across 3 trials per cell.

Generated by scripts/audit-sessions.py. Regenerate after any new trial.

feature

Tool wall-clock min main turns sidechain turns sub-agent disp. files read distinct read files edited tool-config reads target-repo reads cache hit
bmad 21.41 156.33 49.33 1.0 47.0 29.0 10.0 6.0 40.67 0.959
claudekit 97.16 116.0 334.33 2.33 67.33 41.0 47.33 0.0 67.0 0.979
compound 22.0 165.0 53.33 1.67 25.0 17.67 11.67 0.0 25.0 0.977
ecc 634.14 215.67 259.0 1.67 59.0 36.0 56.33 0.0 59.0 0.976
gstack 40.98 155.67 31.0 2.67 21.0 14.33 13.33 0.0 21.0 0.969
omc 223.54 170.67 1556.0 10.67 227.0 63.67 126.33 17.33 209.67 0.928
pure 34.4 138.33 131.33 1.0 45.67 30.33 30.33 0.0 45.67 0.983
superpower 229.57 183.33 778.67 18.33 104.33 52.67 65.67 1.67 101.67 0.946

bugfix

Tool wall-clock min main turns sidechain turns sub-agent disp. files read distinct read files edited tool-config reads target-repo reads cache hit
bmad 9.52 83.33 0.0 0.0 15.33 15.33 3.0 2.67 12.67 0.96
claudekit 21.21 92.67 38.67 0.67 20.67 14.33 2.33 0.0 20.67 0.972
compound 7.22 83.0 0.0 0.0 14.0 13.0 1.0 0.0 14.0 0.972
ecc 22.5 46.0 58.33 1.0 22.67 16.0 1.33 0.0 22.67 0.952
gstack 5.93 72.0 0.0 0.0 12.67 11.67 1.33 0.0 12.67 0.971
omc 37.6 109.0 122.0 3.67 35.33 20.33 7.67 0.33 35.0 0.916
pure 12.56 83.0 25.67 0.33 18.0 14.0 1.33 0.0 17.67 0.965
superpower 13.64 118.33 0.0 0.0 15.33 13.67 4.67 0.0 15.33 0.974

refactor

Tool wall-clock min main turns sidechain turns sub-agent disp. files read distinct read files edited tool-config reads target-repo reads cache hit
bmad 82.11 168.0 70.0 1.33 45.33 27.67 29.0 4.67 40.67 0.959
claudekit 38.16 147.67 19.67 0.33 25.33 17.0 24.67 0.0 25.33 0.983
compound 15.03 165.33 0.0 0.0 22.33 17.0 27.0 0.0 22.0 0.983
ecc 26.3 144.0 41.0 1.0 36.67 20.33 25.0 0.0 36.67 0.975
gstack 46.25 258.67 42.67 2.67 40.33 22.33 32.67 0.0 40.33 0.978
omc 185.03 186.67 481.33 12.33 76.0 36.33 34.0 4.67 71.33 0.926
pure 28.17 143.0 58.33 1.33 42.0 21.67 20.0 0.0 42.0 0.974
superpower 218.47 133.0 42.33 0.67 29.33 18.0 17.67 0.0 29.33 0.952

Skill activation by tool (union across trials, all tasks)

Tool Skills observed (count)
bmad bmad-quick-dev (1380)
claudekit cook (1707), ck-plan (540)
compound compound-engineering:ce-work (727), compound-engineering:ce-plan (387), compound-engineering:ce-code-review (261), compound-engineering:lfg (18), Skill:compound-engineering:ce-plan (9), Skill:compound-engineering:ce-work (8), compound-engineering:ce-commit-push-pr (7), Skill:compound-engineering:ce-code-review (5)
ecc everything-claude-code:plan (427)
gstack autoplan (456), ship (336), investigate (216)
omc oh-my-claudecode:team (2153), oh-my-claudecode:ralplan (1100), oh-my-claudecode:hud (230), oh-my-claudecode:omc-setup (111), Skill:oh-my-claudecode:hud (9), oh-my-claudecode:cancel (6), Skill:oh-my-claudecode:team (1)
pure —
superpower superpowers:subagent-driven-development (2668), superpowers:brainstorming (340), superpowers:systematic-debugging (225), superpowers:test-driven-development (78), superpowers:writing-plans (47), Skill:superpowers:brainstorming (9), Skill:superpowers:writing-plans (3), Skill:superpowers:subagent-driven-development (3)

Sub-agent dispatch by tool (union across trials, all tasks)

Tool sub-agent types observed
bmad Explore (6), general-purpose (1)
claudekit fullstack-developer (4), Explore (4), tester (1), code-reviewer (1)
compound compound-engineering:ce-correctness-reviewer (1), compound-engineering:ce-adversarial-reviewer (1), compound-engineering:ce-api-contract-reviewer (1), compound-engineering:ce-testing-reviewer (1), compound-engineering:ce-maintainability-reviewer (1)
ecc general-purpose (4), everything-claude-code:planner (3), Explore (3), Plan (1)
gstack general-purpose (14), Explore (2)
omc oh-my-claudecode:executor (24), oh-my-claudecode:critic (18), oh-my-claudecode:architect (17), oh-my-claudecode:planner (16), Explore (4), oh-my-claudecode:explore (1)
pure Explore (7), general-purpose (1)
superpower general-purpose (55), Explore (2)

Skill token cost by tool (sum across trials, all tasks)

Output tokens are the cost a skill generated; input + cache_read are the context it consumed. Both are summed across every assistant turn whose attributionSkill matched, joined with that turn’s message.usage. Skills are ranked by output_tokens; only the top 5 per tool are shown.

Tool Skill turns output tok input tok cache_read tok
bmad bmad-quick-dev 1380 693,733 15,003 115,172,822
claudekit cook 1707 525,514 6,473 204,256,027
ck-plan 540 200,432 4,173 34,314,117
compound compound-engineering:ce-work 727 327,751 818 129,836,560
compound-engineering:ce-plan 387 255,051 459 38,803,650
compound-engineering:ce-code-review 261 132,251 10,711 32,426,018
compound-engineering:ce-commit-push-pr 7 5,858 17 1,980,165
compound-engineering:lfg 18 3,698 90 232,688
ecc everything-claude-code:plan 427 137,366 16,333 24,781,278
gstack autoplan 456 349,544 962 33,739,191
ship 336 172,074 418 88,497,292
investigate 216 86,444 246 23,035,833
omc oh-my-claudecode:team 2153 574,078 9,433 217,782,136
oh-my-claudecode:ralplan 1100 559,150 5,066 54,800,217
oh-my-claudecode:hud 230 95,825 1,255 14,829,049
oh-my-claudecode:omc-setup 111 24,450 601 4,716,211
oh-my-claudecode:cancel 6 1,425 29 1,729,537
pure — — — — —
superpower superpowers:subagent-driven-development 2668 669,379 15,796 170,158,857
superpowers:writing-plans 47 162,288 79 4,861,560
superpowers:brainstorming 340 140,225 697 17,888,472
superpowers:systematic-debugging 225 83,967 250 27,368,218
superpowers:test-driven-development 78 34,743 86 9,426,740

Slash commands typed by tool (union across trials, all tasks)

Tool slash commands observed
bmad /bmad-quick-dev (12)
claudekit /ck-plan (9), /cook (9), /copy (2)
compound /compound-engineering:lfg (9), /login (1)
ecc /everything-claude-code:plan (9)
gstack /autoplan (7), /ship (7), /investigate (3)
omc /oh-my-claudecode:team (10), /oh-my-claudecode:omc-setup (9), /clear (9), /oh-my-claudecode:ralplan (9), /login (3), /oh-my-claudecode:cancel (1), /exit (1)
pure /resume (1)
superpower —