seo-audit (coreyhaines31/marketingskills)
coreyhaines31 seo-audit: found no more planted SEO issues than plain Codex (slightly fewer), at 1.2x the tokens
24.7 / 30
Lift over no skill: -0.11
Situational: the lift -0.11 is under +1.0. Recommended needs a total of 22/30, output quality of 3.5, and a lift of +1.0.
SOTA
PassTested with Codex.
Covers Codex, Claude Code
Codex CLI 0.166 (SOTA tier): 18 graded runs. A pass here also covers Claude Code under our tier rule; it says nothing about the second tier.
Scores
Each dimension is 0 to 5 and the six add up to the total out of 30. 5 is better on every dimension. Cost 5 means cheap and fast. Install 5 means easy.
- Trigger reliability
- 5.0
- Lift over no skill
- 1.9
- Output quality
- 4.8
- Token and time cost
- 4.0
- Install and portability
- 4.0
- Safety
- 5.0
Security review
SafeStatic scan: 0 CRITICAL / 2 HIGH / 0 MEDIUM. Manual review: Safe to run in a no-network sandbox (scanner findings explained). A clean scan does not prove a skill safe.
Install
npx skills@latest add coreyhaines31/marketingskills --skill seo-audit
Pinned commit: 88c79f6528a9347420e57ceb1fff9a60de651768
Tested: 2026-11-02
What we tested
- Audit a small static site — Audit the SEO of the static site in this folder (index.html, robots.txt, sitemap.xml; domain https://tidepool.example). Write the findings to audit.md, ordered by severity. Don't change the site files.
- Fix the on-page issues — Fix the on-page SEO problems in index.html and robots.txt (domain https://tidepool.example). Edit the files in place; keep the content otherwise the same.
- Top 5 for a non-technical owner — I'm not technical. Look at the site in this folder and tell me the 5 most important SEO fixes, ranked by impact, in plain language. Do not change any files.
Pros
- The skill was picked up without being named: Skill file read in 6/6 treatment runs.
- High absolute quality with the skill: 4.84 out of 5.
Cons
- No measurable lift over no skill (-0.11, inside the noise).
- A one-sentence prompt scored 4.83, against 4.84 with the skill (+0.01).
- The control already scores 4.95: a ceiling effect, so the test cannot show a lift.
- n=2 per task per arm; differences under about 1 point are noise.
Review
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below). The test site is a small static page set (index.html, robots.txt, sitemap.xml for the fictional domain tidepool.example) with 12 planted issues for the audit task and 9 on-page issues for the fix task, including a Disallow: / robots file and a noindex meta tag. No network, so nothing was fetched.
Three-arm comparison
| Arm | Mean score (0-5) | Tokens per run |
|---|---|---|
| Without the skill | 4.95 | 88k |
| One-sentence prompt | 4.83 | 96k |
| With the skill | 4.84 | 108k |
Measured with Codex CLI over 18 runs, scored by rule-based checks with no LLM judge. Total 24.7 / 30, verdict Situational.
Verdict: Situational (24.7 / 30). Source: coreyhaines31/marketingskills, skills/seo-audit, pinned at commit 88c79f6. Install: npx skills@latest add coreyhaines31/marketingskills --skill seo-audit. Licence: MIT (repo LICENSE).
What it is
The seo-audit skill (v2.2.2) from coreyhaines31/marketingskills. An audit framework for 'my SEO is bad', 'why am I not ranking', technical SEO and meta-tag reviews: check for a product-marketing context file, audit order (crawlability and indexation, technical foundations, on-page, content quality, authority), severity-ranked output, with seven reference files (title tags, international SEO, local SEO, rankings push, AI-writing detection and more). It also says fetched pages are untrusted data. Markdown only.
How we tested
Codex CLI 0.166 (gpt-6-astra, low reasoning) on a Mac test machine, a fresh temporary project and temporary HOME/TMPDIR/caches per run, skill installed (treatment) or absent (control) or absent with the skill's core requirement squeezed into one sentence appended to the prompt (one-liner group), 2 runs per group per task, network off, /tmp writes blocked. The prompts never mention the skill (except the wording noted below).
The test site is a small static page set (index.html, robots.txt, sitemap.xml for the fictional domain tidepool.example) with 12 planted issues for the audit task and 9 on-page issues for the fix task, including a Disallow: / robots file and a noindex meta tag. No network, so nothing was fetched.
- Audit a small static site — Audit the SEO of the static site in this folder (index.html, robots.txt, sitemap.xml; domain https://tidepool.example). Write the findings to audit.md, ordered by severity. Don't change the site files.
- Fix the on-page issues — Fix the on-page SEO problems in index.html and robots.txt (domain https://tidepool.example). Edit the files in place; keep the content otherwise the same.
- Top 5 for a non-technical owner — I'm not technical. Look at the site in this folder and tell me the 5 most important SEO fixes, ranked by impact, in plain language. Do not change any files.
Scoring is programmatic (hidden tests / planted-issue recall / constraint checks / render+OCR where relevant), 0–5 per run. No LLM judge decided any score.
Results
The one-liner given to the third group: "Audit crawlability and indexation (robots.txt, sitemap, canonicals), on-page elements (title, meta description, headings, image alt text, internal links) and structured data, and report findings by severity with a concrete fix for each."
| Task | Control score | One-liner | With skill | Control tokens | One-liner tokens | With-skill tokens | Skill read |
|---|---|---|---|---|---|---|---|
| t1-audit | 4.86 | 4.86 | 4.72 | 105k | 128k | 125k | 2/2 |
| t2-fix | 5.0 | 4.63 | 4.81 | 94k | 94k | 120k | 2/2 |
| t3-priorities | 5.0 | 5.0 | 5.0 | 64k | 63k | 78k | 2/2 |
Mean score: 4.95 without vs 4.84 with the skill (delta -0.11). Tokens per run: 88k vs 108k (1.23x); time 60s vs 62s. One-liner group: 4.83 (the skill's gain over the one-liner: +0.01; one-liner's own gain over no skill: -0.12); one-liner tokens 96k per run, so the skill costs 1.13x the one-liner.
What happened
- No quality gain. Mean score 4.95 without the skill, 4.83 with only a one-sentence version of its audit order, 4.84 with the skill. All three groups reached the planted-issue threshold in every run and the same top finding (robots.txt blocking everything plus the noindex tag) came first.
- In the audit task, planted-issue recall was 11 of 12 in 5 of 6 runs in every group; the missed item varied (missing image alt text, missing viewport tag, thin content). Both skill runs missed 'thin content', the one issue the control got in a run.
- In the fix task, both control runs fixed 9/9 on-page issues; the one-liner runs fixed 8/9 (missed the viewport tag) and the skill runs 8/9 once ('click here' link text left) and 9/9 once. Differences of one issue are noise at n=2.
- The skill was read in 6/6 treatment runs. It cost 108k tokens per run against 88k without it (1.23x) and 96k with the one-liner (1.13x), the smallest overhead of the skills in this round.
- For the non-technical 'top 5 fixes' task all groups scored 5/5 and all put the robots/noindex problem first.
Dimension scores
| Dimension | Score | Why |
|---|---|---|
| 1_output_quality | 4.84 | Mean score 4.84/5 with the skill (programmatic checks, see per-task table). |
| 2_lift_vs_no_skill | 1.89 | -0.11 points vs 4.95 without the skill (n=2 per task; under ~1 point is noise). |
| 3_trigger | 5.0 | Skill file read in 6/6 treatment runs. |
| 4_instructions_portability | 4 | Plain Markdown; optional product-marketing context file and sibling skills (programmatic-seo, schema, ai-seo) referenced but not required. |
| 5_efficiency | 4 | 1.23x tokens (108k vs 88k per run) and 1.03x time (62s vs 60s). |
| 6_safety_hygiene | 5 | Markdown only (scanner HIGH hits were a regex matching 'coast'); includes an explicit rule to treat fetched pages as untrusted data. |
Limits of this test
- A tiny static site with obvious planted problems: a competent agent finds them by reading three files. The skill's strengths (traffic-drop diagnosis, striking-distance keywords, local SEO, international/hreflang, crawl data) need real analytics or a live site and were not tested.
- No network: the skill's page-speed and live-crawl guidance could not run.
- Scoring counts planted issues found by regex; it does not judge prioritisation quality or how well a fix is explained.
- n = 2 per group per task: differences under about 1 point are noise.
- Rule-based scoring measures what the checks can see; it cannot judge taste or whether a result is the best possible.
Security
See security.md. Scan: 0 CRITICAL / 2 HIGH / 0 MEDIUM. Verdict: safe to run (no-network sandbox). A clean scan does not prove a skill safe.
Bottom line
On a small site the skill does not beat plain Codex: both find the major problems and fix the on-page issues, and the skill costs about 1.2x the tokens. The skill is worth keeping for the references it carries (title-tag, local and international SEO) on messier real sites, but this test gives no evidence it makes a basic audit better.
Transcript
Run log for seo-audit: Codex CLI, network off, per-run temp HOME/TMPDIR, one row per graded run. Columns: task, arm-run, score (0-5), tokens, seconds, checks passed, skill files the agent opened. t1-audit control-1: score 4.72, tokens 96625, secs 90.7, checks 3/3, opened nothing t1-audit control-2: score 5.0, tokens 114606, secs 114.4, checks 3/3, opened nothing t1-audit oneliner-1: score 4.72, tokens 119742, secs 95.1, checks 3/3, opened nothing t1-audit oneliner-2: score 5.0, tokens 137601, secs 93.1, checks 3/3, opened nothing t1-audit treatment-1: score 4.72, tokens 139627, secs 111.9, checks 3/3, opened seo-audit t1-audit treatment-2: score 4.72, tokens 110467, secs 100.8, checks 3/3, opened seo-audit t2-fix control-1: score 5.0, tokens 94756, secs 37.3, checks 2/2, opened nothing t2-fix control-2: score 5.0, tokens 94428, secs 36.3, checks 2/2, opened nothing t2-fix oneliner-1: score 4.63, tokens 109171, secs 60.9, checks 2/2, opened nothing
18 runs, 3 artifacts
Sample output
SOTA · With the skill: audit.md (t1-audit, run 1, Codex)
# Tidepool SEO audit Date: 2026-11-02. Intended origin: `https://tidepool.example`. ## Scope and summary Reviewed the complete local `index.html`, `robots.txt`, and `sitemap.xml`, plus README and the folder inventory. No site files were changed. Findings below are ordered by severity; line numbers refer to the original files. The site is not ready for organic search: all crawling is blocked, and the homepage separately requests exclusion from the index. Fix both together before metadata improvements. Next, align HTTPS URLs, repair the sitemap, and verify the missing resources and destinations. The homepage also needs clearer product information and mobile configuration. This is a source audit, not a live crawl. HTTP status codes, redirects, response headers, rendered appearance, search rankings, traffic, and Core Web Vitals were not measured. Missing local files do not prove production 404 responses. The supplied `.example` origin is treated as the audit's intended URL convention, not evidence of a publicly deployed site. Assumed search intent: monthly seaweed snack subscriptions in the United States, based on the supplied copy; keyword demand was not researched. ## Findings, ordered by severity ### 1. Critical — robots.txt blocks the entire site - **Evidence:** `robots.txt:1–2` contains `User-agent: *` and `Disallow: /`. - **Impact:** Compliant crawlers cannot fetch the homepage, its linked pages, or resources. A robots block does not reliably remove a URL from search: externally discovered URLs can still appear without crawled content. - **Recommendation:** For this public marketing site, remove the blanket disallow or replace it with an empty `Disallow:` directive. Coordinate this with finding 2. Verify crawl access after deployment. - **Basis:** [Google robots.txt guidance](https://developers.google.com/search/docs/crawling-indexing/robots/intro). ### 2. Critical — homepage explicitly requests no indexing - **Evidence:** `index.html:6`: `<meta name="robots" content="noindex, follow">`. - **Impact:** Once crawlers can retrieve this page, `noindex` makes it ineligible for indexing. `follow` does not cancel `noindex`. Currently the robots block prevents crawlers from reading this directive, so fixing only one blocker is insufficient. - **Recommendation:** Remove `noindex` from the public homepage; removing the entire tag is sufficient for normal default indexing behavior. After deployment, also check for an `X-Robots-Tag: noindex` response header and use Search Console URL Inspection. - **Basis:** [Google's noindex documentation](https://developers.google.com/search/docs/crawling-indexing/block-indexing). ### 3. High — inconsistent HTTPS and canonical signals - **Evidence:** Both sitemap locations use `http://tidepool.example` (`sitemap.xml:3–4`); the blog link also uses HTTP (`index.html:20`). The HTML head (`index.html:3–8`) contains no canonical link, while the supplied intended origin is HTTPS. - **Impact:** The sitemap and internal link promote a different protocol from the intended URL. If both protocols serve content, this can create duplicate URL discovery and ambiguous consolidation. Missing canonical markup alone is not [... excerpt: first 3200 of 12260 characters]