claude-md.directory · v1.0 frozen eval

Which Claude skill actually helps?

Third-party Brier-audited directory of CLAUDE.md and skills/*.md files. Every entry is scored against a frozen 5-task eval set on claude-haiku-4-5. Reproducible from python scripts/audit.py entries/<slug>.yaml.

7helpful
15neutral
4harmful ⚠️
26audited
⚠️ 3 entries audit as harmful against no-skill baseline on the frozen v1.0 eval set — including 2 of Anthropic's own official skills (algorithmic-art, doc-coauthoring) and Sahil's grow-sustainably. Almost certainly domain mismatch (algorithmic-art is not built for code-review tasks). v1.1 widens to 40 tasks across 8 domains. Author counter-eval slot is open per docs/brier-method.md.
Skill Verdict Brier Baseline Δ Domain Author
Anthropic /algorithmic-art harmful 0.350 0.250 -0.100 spec @anthropics
Anthropic /brand-guidelines neutral 0.300 0.300 +0.000 spec @anthropics
Anthropic /canvas-design neutral 0.250 0.250 +0.000 spec @anthropics
Anthropic /claude-api neutral 0.300 0.300 +0.000 spec @anthropics
Anthropic /doc-coauthoring harmful 0.400 0.250 -0.150 spec @anthropics
Anthropic /docx neutral 0.300 0.300 +0.000 spec @anthropics
Anthropic /frontend-design helpful 0.300 0.450 +0.150 spec @anthropics
Anthropic /internal-comms helpful 0.300 0.450 +0.150 spec @anthropics
Anthropic /mcp-builder neutral 0.250 0.300 +0.050 spec @anthropics
Anthropic /pdf neutral 0.300 0.250 -0.050 spec @anthropics
Anthropic /pptx neutral 0.300 0.250 -0.050 spec @anthropics
Anthropic /skill-creator neutral 0.300 0.250 -0.050 spec @anthropics
Anthropic /slack-gif-creator neutral 0.250 0.250 +0.000 spec @anthropics
Anthropic /theme-factory neutral 0.300 0.300 +0.000 spec @anthropics
Anthropic /web-artifacts-builder neutral 0.250 0.250 +0.000 spec @anthropics
Karpathy's CLAUDE.md (4 behavioral rules) helpful 0.250 0.500 +0.250 write @multica-ai
Sahil /company-values helpful 0.050 0.250 +0.200 spec @slavingia
Sahil /find-community neutral 0.300 0.300 +0.000 spec @slavingia
Sahil /first-customers helpful 0.250 0.400 +0.150 spec @slavingia
Sahil /grow-sustainably neutral 0.250 0.250 +0.000 spec @slavingia
Sahil /marketing-plan helpful 0.250 0.400 +0.150 spec @slavingia
Sahil /minimalist-review harmful 0.450 0.250 -0.200 review @slavingia
Sahil /mvp neutral 0.250 0.250 +0.000 spec @slavingia
Sahil /pricing neutral 0.450 0.450 +0.000 spec @slavingia
Sahil /processize helpful 0.050 0.450 +0.400 ship @slavingia
Sahil /validate-idea harmful 0.400 0.250 -0.150 spec @slavingia