Free robots.txt Generator
Choose which AI scrapers you want to allow access to your site to generate and download your free robots.txt document
Configure Bot Access
Toggle which AI models are allowed to crawl your content guidelines.
🤖 GPTBot
🤖 ClaudeBot
🤖 PerplexityBot
🤖 Google-Extended
🌍 Global Rule (All Other Crawlers)
How to upload your robots.txt file in Framer
How to upload your robots.txt file in Webflow
How to upload your robots.txt file in WordPress
How to upload your robots.txt file in Shopify
How to upload your robots.txt file in Squarespace
How to upload your robots.txt file in Wix
Why should I include a sitemap in my robots.txt file?
Product
Resources
Articles & Guides
Documentation
Compare
Meerkat AI vs Otterly
Meerkat AI vs Geoptie
Meerkat AI vs Searchable
Meerkat AI vs Peek AI
Meerkat AI vs Rankscale
© 2026 Meerkat AI. All rights reserved.