Content Cucumber
The robots.txt + llms.txt
Punch List
Five steps. Copy-paste templates. AI-ready in an afternoon. Follow this checklist and end up ahead of 90% of sites.
- Find out in 60 seconds whether your site is invisible to AI search or already ahead of the curve
- Copy the robots.txt template that names every major AI crawler: GPTBot, ClaudeBot, PerplexityBot, Amazonbot, Applebot-Extended, and more
- See the allow-search vs block-training decision most sites get wrong by default, with the one-line rule you want
- Get a spec-compliant llms.txt template that AI systems can parse, plus the six mistakes that break the file for everyone else
- Learn the before/after test that shows how ChatGPT, Claude, and Perplexity answer 'what is your company?' once the files go live
- Build a quarterly maintenance cadence so new AI crawlers never catch you flat-footed again
Get the Punch List
Free. Checklist plus templates. Delivered to your inbox.