Building evaluation suites that decide between models for a task: what the eval measures before any prompt is written, constructing cases from real failure modes and coding them from task-native artifacts, contamination and why held-out data no longer guarantees freshness, automatic scoring with exact-match and judge limits, pass-at-k and aggregate metrics read honestly, human comparison and preference data as a separate instrument, and the written evaluation record. It designs the measurement; it never certifies a model as good and never recommends a vendor. Use when asking "model evaluation", "eval suite design", "build a benchmark", "contamination", "held-out set", "LLM as judge", "pass at k", "model comparison", "regression suite for model upgrades", "golden set", "cost of evaluation". Not for improving a prompt against fixed cases (ac-prompt-engineer), not for running the harness code itself (ac-llm-api-integration), not for evaluating datasets as research evidence (ac-alternative-data).
cp -r cryptoskill/skills/exchanges/anoscranel-ac-model-evaluation .claude/skills/clawhub install anoscranel-ac-model-evaluationCapabilities below are detected automatically by an open-source scanner that reads the skill's text and scripts (see how this is computed). Not measured means the scanner couldn't make a confident call — it is NOT a green check, and you should treat it as a possible red flag until a human or a stronger scanner has measured it.
analysis_only Phase 1 single-mode classification — multi-mode breakdown deferred to Phase 2.
External services this skill needs in order to work. The skill's behavior depends on these services being up, honest, and unchanged. Today we only show services we recognize from a curated list of ~50 well-known hosts (exchanges, RPC providers, data APIs); a complete dependency list — including every package, library, and binary the skill installs or imports — is on the roadmap.
anthropic.comNo one has audited this skill yet. That is different from “audited and clean” — it just means no professional reviewer (a security firm, the CryptoSkill team, or a verified independent researcher) has signed off on it. There are no audit reports to read. How reviewer levels work →
SKILL.md → SOURCE.md → TRUST.auto.yaml → Browse directory →
Auto-generated by cryptoskill/extract-capabilities/0.3.1 · hosted-service list version 2026-09-06 · how this is computed