Content Cucumber

The robots.txt + llms.txt
Punch List

Five steps. Copy-paste templates. AI-ready in an afternoon. Follow this checklist and end up ahead of 90% of sites.

  • Find out in 60 seconds whether your site is invisible to AI search or already ahead of the curve
  • Copy the robots.txt template that names every major AI crawler: GPTBot, ClaudeBot, PerplexityBot, Amazonbot, Applebot-Extended, and more
  • See the allow-search vs block-training decision most sites get wrong by default, with the one-line rule you want
  • Get a spec-compliant llms.txt template that AI systems can parse, plus the six mistakes that break the file for everyone else
  • Learn the before/after test that shows how ChatGPT, Claude, and Perplexity answer 'what is your company?' once the files go live
  • Build a quarterly maintenance cadence so new AI crawlers never catch you flat-footed again
The robots.txt + llms.txt<br><em>Punch List</em>
Content Cucumber writes for brands that want AI systems to represent them accurately. This punch list is the same checklist we work through before publishing any client's robots.txt and llms.txt.

Get the Punch List

Free. Checklist plus templates. Delivered to your inbox.