Free Tool

Robots.txt Generator

Build a crawl-rule file in seconds.

Default Rule

Sitemap URLBlock AI training crawlers (GPTBot, ClaudeBot, CCBot, Google-Extended)

robots.txt

User-agent: *
Allow: /

Sitemap: https://yoursite.com/sitemap.xml