emails (coreyhaines31/marketingskills)

Review

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below). Three writing tasks with a fact sheet (product.md, a fictional seaweed-snack subscription): a 5-email welcome sequence, a 3-email abandoned-cart flow (no discounts allowed) and a win-back email with 5 alternative subject lines. Each has hard limits (subject max 50 characters, body max 100-120 words, exactly one CTA). Scored by rules: file exists, limits kept, no invented numbers, discounts or urgency, no banned buzzwords, no AI-sounding patterns (it's-not-X-it's-Y, no-X-no-Y-no-Z, fake 'Re:' subjects).

Three-arm comparison

Measured with Codex CLI over 18 runs, scored by rule-based checks with no LLM judge. Total 24.0 / 30, verdict Situational.

Verdict: Situational (24.0 / 30). Source: coreyhaines31/marketingskills, skills/emails, pinned at commit 88c79f6. Install: npx skills@latest add coreyhaines31/marketingskills --skill emails. Licence: MIT (repo LICENSE).

What it is

The emails skill from coreyhaines31/marketingskills: designs email sequences (welcome, nurture, abandoned cart, win-back): goals per email, subject lines and preview text, copy rules, timing and send order, with three reference files (copy guidelines, email types, sequence templates). About 1,650 words in SKILL.md and 3,200 in the references. Markdown only, no scripts.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Three writing tasks with a fact sheet (product.md, a fictional seaweed-snack subscription): a 5-email welcome sequence, a 3-email abandoned-cart flow (no discounts allowed) and a win-back email with 5 alternative subject lines. Each has hard limits (subject max 50 characters, body max 100-120 words, exactly one CTA). Scored by rules: file exists, limits kept, no invented numbers, discounts or urgency, no banned buzzwords, no AI-sounding patterns (it's-not-X-it's-Y, no-X-no-Y-no-Z, fake 'Re:' subjects).

1. 5-email welcome sequence — Write a 5-email welcome sequence for new Tidepool subscribers (facts in product.md only). For each email give 'Subject:' (max 50 characters), 'Preview:' text, a body of at most 120 words and exactly one CTA. Save to welcome.md.
2. 3-email abandoned-cart flow — Write a 3-email abandoned-cart sequence for Tidepool (facts in product.md only; there are no discounts or guarantees, so don't offer any). Each email: 'Subject:' (max 50 characters), 'Preview:', body at most 100 words, one CTA. Save to cart.md.
3. Win-back email + subject variants — Subscribers who skipped 3 months in a row haven't ordered. Write one re-engagement email (Subject:, Preview:, body max 100 words, one CTA) and 5 alternative subject lines (each max 50 characters). Use only facts from product.md. Save to winback.md.

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "Design each email with one clear goal and exactly one call to action, a short specific subject line and preview text, plain human tone with no AI tell-tale patterns, and a sensible send order and timing across the sequence."

Mean score: 5.0 without vs 5.0 with the skill (delta +0.0). Tokens per run: 69k vs 102k (1.48x); time 38s vs 50s. One-liner group: 5.0 (the skill's gain over the one-liner: +0.0; one-liner's own gain over no skill: +0.0); one-liner tokens 94k per run, so the skill costs 1.09x the one-liner.

What happened

• No difference on the rules: mean score 5.0 for control, 5.0 for the one-sentence version, 5.0 for the skill. Every email in all 18 runs met every limit. Plain Codex already follows an explicit fact sheet, length limits and 'exactly one CTA'.
• A scoring fix was needed: the first pass counted send timings ('Send: 10 days after subscribing', '24 hours after abandonment') as invented numbers and gave the one-liner and skill groups 4.6 and 4.9. Timing is scheduling advice, not a product claim, so those phrases are now ignored; the first scores are not used.
• The one visible difference is the timing guidance the skill talks about. Looking for a send-time label (a 'Send:' line or a timing in the email heading), control wrote one in 0 of 6 sequences, the one-sentence version in 4 of 6, and the skill in 1 of 6. So the skill did not reliably add timing, but the sentence that names it did.
• The skill was read in 6/6 runs and cost 102k tokens per run against 69k for control (1.48x) and 94k for the one-liner (1.09x).

Dimension scores

Limits of this test

• A ceiling effect: control is already at 5.0, so the test cannot show improvement; it shows only that nothing got worse.
• The checks cover limits, honesty and style tells, not persuasion quality, subject-line cleverness or deliverability; a human reading of copy quality was not done beyond spot checks.
• Only a fictional single-product brief; the skill's value for nurture and B2B drip sequences with real segmentation data was not tested.
• n = 2 per group per task: differences under about 1 point are noise.
• Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

Plain Codex produces compliant, fact-based email sequences without the skill, and the skill adds tokens (about 1.5x) with no measurable change in these tasks. It may help when you want the skill's structure and timing guidance in the output, which one sentence in the prompt also achieved more often here.