tdd (Matt Pocock)

Name: tdd (Matt Pocock) review
Item: tdd (Matt Pocock)
Rating: 22.7
Author: Agent Skills

mattpocock tdd: no lift once the prompt already asks for test-first, and about 1.7x the tokens

SituationalSafe

22.7 / 30

Lift over no skill: +0.00

Situational: the lift +0.00 is under +1.0. Recommended needs a total of 22/30, output quality of 3.5, and a lift of +1.0.

SOTA

Pass

Tested with Codex.

Covers Codex, Claude Code

Codex CLI 0.166 (SOTA tier): 18 graded runs. A pass here also covers Claude Code under our tier rule; it says nothing about the second tier.

Scores

Each dimension is 0 to 5 and the six add up to the total out of 30. 5 is better on every dimension. Cost 5 means cheap and fast. Install 5 means easy.

Trigger reliability: 5.0
Lift over no skill: 2.0
Output quality: 4.7
Token and time cost: 2.0
Install and portability: 4.0
Safety: 5.0

Security review

Safe

Static scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Manual review: Safe to run in a no-network sandbox (no findings). A clean scan does not prove a skill safe.

Install

npx skills@latest add mattpocock/skills --skill tdd

Source

Pinned commit: 4ae2317ddb22b046e26455e4f9c9b3aaaf62212d

Tested: 2026-11-02

What we tested

Implement a function from a precise spec — Implement slugify(text, max_length=None) in slug.py (Python 3, standard library only) using TDD. Spec: lowercase the text; replace each run of characters that are not ASCII letters or digits with a single '-'; strip leading and trailing '-'; an empty result…
Implement a small stateful class — Write ratelimiter.py (Python 3, standard library only) test-first (red-green-refactor) with class RateLimiter(limit, window_seconds, clock=time.monotonic): a sliding-window limiter per key. allow(key) returns True and records the call if fewer than limit…
Fix a rounding bug — Customers say cart_total in cart.py is sometimes a cent off compared with our accounting system, which rounds half-up to the cent at each step (each line total, the discounted subtotal, and the tax). For example cart_total([(1.005, 1)]) should return 1.09 an…

Pros

The skill was picked up without being named: Skill file read in 6/6 treatment runs.
High absolute quality with the skill: 4.67 out of 5.

Cons

No measurable lift over no skill (+0.00, inside the noise).
A one-sentence prompt scored 4.67, against 4.67 with the skill (+0.00).
Uses 1.7x the tokens of no skill.
Test caveat: The one-sentence arm (6 runs) was added later on a different machine from the control and treatment arms.
n=2 per task per arm; differences under about 1 point are noise.

Review

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below). Because this skill's description only triggers when the user asks for TDD / test-first, the prompts say so ("using TDD", "test-first (red-green-refactor)", "Do it test-first"). The same words are given to the control group, so the test is whether the skill improves on a plain request for TDD.

Three-arm comparison

Arm	Mean score (0-5)	Tokens per run
Without the skill	4.67	86k
One-sentence prompt	4.67	178k
With the skill	4.67	146k

Measured with Codex CLI over 18 runs, scored by rule-based checks with no LLM judge. Total 22.7 / 30, verdict Situational.

Disclosure: The one-sentence arm (6 runs) was added later on a different machine from the control and treatment arms.

Verdict: Situational (22.7 / 30). Source: mattpocock/skills, skills/engineering/tdd, pinned at commit 4ae2317. Install: npx skills@latest add mattpocock/skills --skill tdd. Licence: MIT (repo LICENSE).

What it is

The tdd skill from Matt Pocock's skills repo (engineering/tdd): the red-green loop plus two reference files on what a good test is and when to mock. About 570 words in SKILL.md; pure Markdown, no scripts. It triggers when the user wants test-first work.

How we tested

Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).

Because this skill's description only triggers when the user asks for TDD / test-first, the prompts say so ("using TDD", "test-first (red-green-refactor)", "Do it test-first"). The same words are given to the control group, so the test is whether the skill improves on a plain request for TDD.

Implement a function from a precise spec — Implement slugify(text, max_length=None) in slug.py (Python 3, standard library only) using TDD. Spec: lowercase the text; replace each run of characters that are not ASCII letters or digits with a single '-'; strip leading and trailing '-'; an empty result…
Implement a small stateful class — Write ratelimiter.py (Python 3, standard library only) test-first (red-green-refactor) with class RateLimiter(limit, window_seconds, clock=time.monotonic): a sliding-window limiter per key. allow(key) returns True and records the call if fewer than limit…
Fix a rounding bug — Customers say cart_total in cart.py is sometimes a cent off compared with our accounting system, which rounds half-up to the cent at each step (each line total, the discounted subtotal, and the tax). For example cart_total([(1.005, 1)]) should return 1.09 an…

Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.

Results

The one-liner given to the third group: "Work test-first in vertical slices: write one failing test through the public interface, confirm it fails, write the minimum code to pass it, repeat, and refactor only when green; do not write all the tests up front."

Task	Control score	One-liner	With skill	Control tokens	One-liner tokens	With-skill tokens	Skill read
t1-slugify	5.0	5.0	5.0	79k	184k	135k	2/2
t2-ratelimiter	5.0	5.0	5.0	99k	183k	169k	2/2
t3-cart-bugfix	4.0	4.0	4.0	79k	164k	133k	2/2

Mean score: 4.67 without vs 4.67 with the skill (delta +0.0). Tokens per run: 86k vs 146k (1.70x); time 50s vs 67s. One-liner group: 4.67 (the skill's gain over the one-liner: +0.0; one-liner's own gain over no skill: +0.0); one-liner tokens 178k per run, so the skill costs 0.82x the one-liner.

What happened

With the prompt already asking for test-first, the control group did it too: on the two build tasks every run of both groups passed all checks, including 'test written before implementation'. The skill added nothing measurable there.
On the bug-fix task (cart rounding) all four runs, with and without the skill, fixed the bug and passed the hidden tests, and all four failed the same process check: the fix or the test arrived in the same step as the other, rather than a failing test first. The skill did not change that.
The skill was read in 6/6 treatment runs. It made the agent do far more work (24-26 commands vs 8-10 on the build tasks) for the same result: about 1.7x the tokens.
Hidden-test correctness was 100% in both groups.
One-sentence version (work test-first in vertical slices, one failing test at a time): mean score 4.67, identical to control and skill, at 178k tokens per run against 86k for control (2.1x) and 146k for the skill. The one-liner agents ran 27 commands per run on average against 9 for control and 23 for the skill, because they ran the test loop repeatedly.
Every group wrote its own tests in all runs and passed all hidden tests; tests-before-implementation was met in 4 of 6 runs in all three groups, so the prompt-level request 'test first' already did what the skill is for.

Dimension scores

Dimension	Score	Why
1_output_quality	4.67	Mean score 4.67/5 with the skill (programmatic checks, see per-task table).
2_lift_vs_no_skill	2.0	+0.00 points vs 4.67 without the skill (n=2 per task; under ~1 point is noise).
3_trigger	5.0	Skill file read in 6/6 treatment runs.
4_instructions_portability	4
5_efficiency	2	1.70x tokens (146k vs 86k per run) and 1.34x time (67s vs 50s).
6_safety_hygiene	5

Limits of this test

Each test prompt itself asked for TDD, which gives the control group most of the benefit; a user who does not ask would see the skill's description not trigger at all.
Small Python tasks only; test quality was not scored.
n = 2 per group per task: differences under about 1 point are noise.
Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.

Security

See security.md. Scan: 0 CRITICAL / 0 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.

Bottom line

On top of a test-first request this skill produces no extra gain on small tasks: control, the one-sentence version and the skill all scored 4.67, while the skill used 1.7x and the sentence 2.1x the tokens of control. It may help keep tests behaviour-focused over a longer session, which we did not measure.

Transcript

Run log for tdd: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run.
Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened.

t1-slugify control-1: score 5.0, tokens 78904, secs 50.3, checks 3/3, opened nothing
t1-slugify control-2: score 5.0, tokens 79205, secs 47.1, checks 3/3, opened nothing
t1-slugify oneliner-1: score 5.0, tokens 199600, secs 74.7, checks 3/3, opened nothing
t1-slugify oneliner-2: score 5.0, tokens 168650, secs 68.2, checks 3/3, opened nothing
t1-slugify treatment-1: score 5.0, tokens 144538, secs 57.6, checks 3/3, opened tdd
t1-slugify treatment-2: score 5.0, tokens 126047, secs 50.7, checks 3/3, opened tdd
t2-ratelimiter control-1: score 5.0, tokens 99439, secs 60.2, checks 3/3, opened nothing
t2-ratelimiter control-2: score 5.0, tokens 99611, secs 54.3, checks 3/3, opened nothing
t2-ratelimiter oneliner-1: score 5.0, tokens 159969, secs 70.9, checks 3/3, opened nothing

18 runs, 3 artifacts

Sample output

SOTA · With the skill: test_slug.py (t1-slugify, run 1, Codex)

import unittest from slug import slugify class SlugifyTests(unittest.TestCase): def test_hard_cuts_first_word_when_it_exceeds_limit(self): self.assertEqual(slugify('Supercalifragilistic', 5), 'super') def test_truncates_at_word_boundary(self): self.assertEqual(slugify('Hello, World!', 8), 'hello') def test_normalizes_text(self): self.assertEqual(slugify('Hello, World!'), 'hello-world') def test_preserves_only_ascii_letters_and_digits(self): self.assertEqual(slugify(' -- CAFÉ_42\t中文 / Test!! '), 'caf-42-test') def test_lowercases_before_filtering(self): self.assertEqual(slugify('\u212aELVIN'), 'kelvin') def test_keeps_slug_that_fits_limit(self): for limit in (11, 20, None): with self.subTest(limit=limit): self.assertEqual(slugify('Hello, World!', limit), 'hello-world') def test_keeps_complete_words_without_trailing_separator(self): for limit, expected in ((3, 'one'), (4, 'one'), (6, 'one'), (7, 'one-two'), (8, 'one-two')): with self.subTest(limit=limit): self.assertEqual(slugify('one two three', limit), expected) def test_hard_cuts_long_first_word_even_with_later_words(self): self.assertEqual(slugify('Supercalifragilistic hello', 5), 'super') def test_applies_limit_to_fallback(self): self.assertEqual(slugify('', 2), 'n') def test_empty_normalized_text_has_fallback(self): for text in ('', ' !!! ', '中文'): with self.subTest(text=text): self.assertEqual(slugify(text), 'n-a') if __name__ == '__main__': unittest.main()

See Pro