Robots.txt Guides for Every Platform
Platform-specific examples, templates, and best practices — pick your stack and follow the guide.
Every guide includes a copy-ready template, a "Test this robots.txt →" shortcut, common mistakes, and an FAQ.
By Platform
Configuration guides for popular CMS platforms, hosted builders, and site types.
Robots.txt for WordPress Block /wp-admin/, /wp-includes/, /xmlrpc.php, and search results. Includes plugin-based editing guide.Robots.txt for Shopify Shopify's built-in file is locked. Learn how to request custom rules and what you can control.Robots.txt for Wix Use Wix's built-in editor to block admin routes and add your sitemap directive.Robots.txt for eCommerce Protect checkout pages, accounts, and internal search while keeping product pages open.Robots.txt for Blog Block tag and author archives while keeping your posts, feeds, and categories open to crawlers.Robots.txt for Staging Site Use Disallow: / to block all crawlers from indexing your staging environment.
By Tech Stack
Framework-specific guides covering where to place the file, what to block, and how to test.
Robots.txt for Next.js Block /_next/ build artifacts and /api/ routes. Covers /public/ placement and app/robots.ts.Robots.txt for React Place in /public/ for Vite/CRA. Includes SPA SEO limitations and sitemap generation tips.Robots.txt for Laravel Block /storage/, /vendor/, /telescope/, and /horizon/. Must be in /public/, not the project root.Robots.txt for Angular Place in /src/ so ng build includes it. Covers Angular SSR for better indexing.Robots.txt for Static Sites Netlify, Vercel, and GitHub Pages setup. Keep rules minimal — everything is pre-rendered.
Not sure where to start?
Use our generator to build the right robots.txt for your site — then test it instantly to check which URLs are blocked or allowed.