theme-factory (Anthropic)
theme-factory: Codex already knows two named themes by heart (5.0 without the skill), but only the skill can 'pick a preset': 5.0 vs 0.0 on that task, and it is cheaper (0.66x tokens)
27.7 / 30
Lift over no skill: +1.67
Recommended: the total 27.7/30 is at least 22, output quality 5.0 is at least 3.5, and the lift +1.67 is at least +1.0.
SOTA
PassTested with Codex.
Covers Codex, Claude Code
Codex CLI 0.166 (SOTA tier): 18 graded runs. A pass here also covers Claude Code under our tier rule; it says nothing about the second tier.
Scores
Each dimension is 0 to 5 and the six add up to the total out of 30. 5 is better on every dimension. Cost 5 means cheap and fast. Install 5 means easy.
- Trigger reliability
- 5.0
- Lift over no skill
- 3.7
- Output quality
- 5.0
- Token and time cost
- 5.0
- Install and portability
- 4.0
- Safety
- 5.0
Security review
SafeStatic scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Manual review: Safe to run in a no-network sandbox (no findings). A clean scan does not prove a skill safe.
Install
npx skills@latest add anthropics/skills --skill theme-factory
Pinned commit: ca45fc496a3436251f191354091d557da31c15b5
Tested: 2026-11-02
What we tested
- Apply a named preset theme — Apply the 'Ocean Depths' theme to deck.html. Edit the file in place.
- Apply another preset theme — Apply the 'Tech Innovation' theme to deck.html. Edit the file in place.
- Pick a fitting preset — Pick the preset theme that best fits a children's science fair announcement, state which one you chose, and apply it to deck.html in place.
Pros
- Lifts the mean score from 3.33 to 5.0 over no skill (+1.67).
- The skill was picked up without being named: Skill file read in 6/6 treatment runs.
- No token overhead: 0.66x the tokens of no skill.
- Beats a one-sentence version of its own rule by +1.67.
- High absolute quality with the skill: 5.0 out of 5.
Cons
- Test caveat: The control group knows named themes from memory; only the skill can pick from its list, which explains the large gap against the one-sentence arm.
- n=2 per task per arm; differences under about 1 point are noise.
Review
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below). Three tasks on the same plain HTML slide deck (deck.html): apply the named theme 'Ocean Depths', apply the named theme 'Tech Innovation', and 'pick the preset theme that best fits a children's science fair, state which one, and apply it'. The one-liner group adds one sentence telling the agent to apply a preset theme's palette and font pairing and say which one; it contains no theme data.
Three-arm comparison
| Arm | Mean score (0-5) | Tokens per run |
|---|---|---|
| Without the skill | 3.33 | 111k |
| One-sentence prompt | 3.33 | 126k |
| With the skill | 5.0 | 73k |
Measured with Codex CLI over 18 runs, scored by rule-based checks with no LLM judge. Total 27.7 / 30, verdict Recommended.
Disclosure: The control group knows named themes from memory; only the skill can pick from its list, which explains the large gap against the one-sentence arm.
Verdict: Recommended (27.7 / 30). Source: anthropics/skills, skills/theme-factory, pinned at commit ca45fc4. Install: npx skills@latest add anthropics/skills --skill theme-factory. Licence: Apache-2.0 / Anthropic terms (see LICENSE.txt in skill).
What it is
The theme-factory skill from anthropics/skills: 10 preset colour-and-font themes (Ocean Depths, Sunset Boulevard, Forest Canopy, Tech Innovation and so on), each a short markdown file with hex palette and font pairing, plus a theme showcase PDF, for styling slides, docs or pages. SKILL.md tells the agent to show the showcase, let the user choose, then apply the theme's colours and fonts.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a dedicated Windows test PC, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
Three tasks on the same plain HTML slide deck (deck.html): apply the named theme 'Ocean Depths', apply the named theme 'Tech Innovation', and 'pick the preset theme that best fits a children's science fair, state which one, and apply it'. The one-liner group adds one sentence telling the agent to apply a preset theme's palette and font pairing and say which one; it contains no theme data.
- Apply a named preset theme — Apply the 'Ocean Depths' theme to deck.html. Edit the file in place.
- Apply another preset theme — Apply the 'Tech Innovation' theme to deck.html. Edit the file in place.
- Pick a fitting preset — Pick the preset theme that best fits a children's science fair announcement, state which one you chose, and apply it to deck.html in place.
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Apply the chosen preset theme's color palette (background, text, accent) and its header/body font pairing consistently to every element, and say which theme you applied."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-ocean | 5.0 | 5.0 | 5.0 | 70k | 103k | 69k | 2/2 |
| t2-tech | 5.0 | 5.0 | 5.0 | 73k | 102k | 79k | 2/2 |
| t3-choose | 0.0 | 0.0 | 5.0 | 187k | 171k | 71k | 2/2 |
Mean score: 3.33 without vs 5.0 with the skill (delta +1.67). Tokens per run: 111k vs 73k (0.66x); time 53s vs 44s. One-liner group: 3.33 (the skill's gain over the one-liner: +1.67; one-liner's own gain over no skill: +0.0); one-liner tokens 126k per run, so the skill costs 0.58x the one-liner.
What happened
- On the two named-theme tasks all three groups score 5.0: control (no skill, no theme files in the folder) applied the exact Ocean Depths hexes (#1a2332, #2d8b8b, #a8dadc, #f1faee) and Tech Innovation palette from memory. The preset themes are public enough that the model already knows them; the skill adds nothing on a request that names the theme.
- The skill matters where the theme list itself is needed: on the 'pick the best preset' task control scored 0/2 (one run said it chose a preset called 'Sky', which is not one of the ten; the other spent 247k tokens), the one-liner scored 0/2 (one run replied that no preset list was available in the workspace and asked the user for it), and the skill scored 2/2 (picked 'Sunset Boulevard' and applied its palette).
- Mean 3.33 control, 3.33 one-liner, 5.00 skill. The one-liner cannot substitute because the value is the data in the skill's theme files, not an instruction.
- Tokens: 73k per run for the skill against 111k control and 126k one-liner (0.66x and 0.58x), because on the open task control and one-liner search and guess while the skill reads a small theme file.
- Treatment runs copy the skill's themes folder and showcase PDF into the project (14 new files per run on average), which pollutes the working folder.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 5.0 | Mean score 5.00/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 3.67 | +1.67 points vs 3.33 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | SKILL.md of 43 lines, 10 theme files, a PDF showcase; no scripts. |
| 5_efficiency | 5 | 0.66x tokens (73k vs 111k per run) and 0.83x time (44s vs 53s). |
| 6_safety_hygiene | 5 | Prose, data and a PDF only. |
Limits of this test
- Rule-based palette checks (preset hex colours used, theme named); not a judgement of how good the restyled deck looks.
- Task 3's check counts a run as correct only if it uses one of the ten themes' hexes and names one, so it favours any group that can see the list.
- n=2 per cell; the skill's showcase step (showing the PDF to the user and waiting for a choice) is interactive and cannot run in a headless test.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
Useful when you want one of the ten specific presets chosen or listed: it removes guessing and saves tokens. If you just name a preset, plain Codex already does the same, so the skill is not needed for that.
Transcript
Run log for theme-factory: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-ocean control-1: score 5.0, tokens 74123, secs 43.0, checks 2/2, opened nothing t1-ocean control-2: score 5.0, tokens 66741, secs 48.1, checks 2/2, opened nothing t1-ocean oneliner-1: score 5.0, tokens 112572, secs 60.5, checks 2/2, opened nothing t1-ocean oneliner-2: score 5.0, tokens 94515, secs 51.3, checks 2/2, opened nothing t1-ocean treatment-1: score 5.0, tokens 69081, secs 47.0, checks 2/2, opened theme-factory t1-ocean treatment-2: score 5.0, tokens 70216, secs 33.6, checks 2/2, opened theme-factory t2-tech control-1: score 5.0, tokens 77803, secs 56.7, checks 1/1, opened nothing t2-tech control-2: score 5.0, tokens 69182, secs 57.9, checks 1/1, opened nothing t2-tech oneliner-1: score 5.0, tokens 119920, secs 57.6, checks 1/1, opened nothing
18 runs, 3 artifacts
Sample output
SOTA · With the skill: deck.html (t1-ocean, run 1, Codex)
<!DOCTYPE html> <html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>Q3 Update</title> <style> :root { color-scheme: dark; --navy: #1a2332; --teal: #2d8b8b; --seafoam: #a8dadc; --cream: #f1faee; } * { box-sizing: border-box; } body { margin: 0; background: var(--navy); color: var(--cream); font-family: "DejaVu Sans", sans-serif; font-size: clamp(18px, 2.2vw, 28px); line-height: 1.6; } section { max-width: 1200px; min-height: 100vh; margin: 0 auto; padding: clamp(32px, 7vw, 96px); display: flex; flex-direction: column; justify-content: center; border-top: 6px solid var(--teal); } h1, h2 { font-family: "DejaVu Sans", sans-serif; font-weight: 700; line-height: 1.15; margin: 0 0 0.6em; } h1 { font-size: clamp(48px, 7vw, 88px); } h2 { color: var(--seafoam); font-size: clamp(36px, 5vw, 60px); } p { margin: 0 0 1em; } section:first-child p { color: var(--seafoam); } ul { margin: 0; padding-left: 1.3em; } li { padding: 0.25em 0; } li::marker { color: var(--seafoam); } table { width: 100%; border-collapse: collapse; border: 2px solid var(--teal); margin-top: 1em; } th, td { padding: 0.65em 1em; text-align: left; border: 1px solid var(--teal); } th { background: var(--seafoam); color: var(--navy); font-weight: 700; } @media print { section { min-height: 0; height: 100vh; break-after: page; } section:last-child { break-after: auto; } body { print-color-adjust: exact; -webkit-print-color-adjust: exact; } } </style> </head> <body> <section><h1>Q3 Update</h1><p>Prepared for the leadership team</p></section> <section><h2>Highlights</h2><ul><li>Revenue up 12%</li><li>Two launches shipped</li><li>Support backlog cleared</li></ul></section> <section><h2>Next quarter</h2><p>Focus: reliability, onboarding, pricing review.</p><table border="1"><tr><th>Metric</th><th>Q3</th></tr><tr><td>Revenue</td><td>$1.2M</td></tr><tr><td>Churn</td><td>2.1%</td></tr></table></section> </body></html>