Free LLMs.txt Generator
This generator writes a correctly formatted llms.txt file from your own answers: AI crawler controls, attribution settings, and editable allow and block paths. It runs in your browser and never crawls your site, so the output reflects what you tell it. Review the paths before uploading the file to your root.
Enter Your Website Details
Your answers below build the file. Nothing here visits your site — everything is generated in your browser from what you enter.
Used to label the file. Your site is not visited or read.
Used in your llms.txt for crawler communication. Never shared.
Could not generate llms.txt
Check the URL and try again.
Your llms.txt is ready!
Upload this file to your website root as llms.txt
Built from your answers, not a crawl
This file was generated in your browser from the details you typed and the standard AI crawler directives. It is not live data from your website, Google, or a third-party provider — no request left this page, so the paths below are suggestions rather than verified URLs. Confirm each Allow and Disallow line matches your real site before you upload it.
Your Settings
Generated LLMs.txt
Format
This confirms the file's syntax, not that the paths it lists exist on your site.
What is llms.txt?
llms.txt is an emerging standard that lets websites control how AI crawlers access their content. Fill in the details above and the tool writes a correctly formatted llms.txt with the AI crawler directives you choose. It builds the file from your answers in your browser — it does not visit or read your site, so review the suggested paths before uploading.
How It Works
- Enter Website Details — Type your URL, contact email, and site description. These go into the file itself, so the result is specific to you rather than a blank template.
- Map Your Content — Tick the kinds of content your site has and the tool proposes the conventional path for each one (/blog/, /docs/, /products/, /admin/ and so on). Nothing is fetched or discovered.
- Customize Rules — Review the suggested sections against your real URLs, toggle individual AI crawlers, and add custom allow/block paths for fine-grained control.
- Copy or Download — Get your llms.txt in the standard format. Check each path, then upload it to yourdomain.com/llms.txt.
Use Cases by Site Type
- Blogs & Content Sites — Allow AI crawlers to access articles and guides while blocking admin areas. Require attribution to protect your intellectual property.
- E-commerce Stores — Allow product descriptions and blog content for AI search visibility. Block checkout, cart, and account pages.
- SaaS Platforms — Allow documentation and public marketing pages. Block app dashboards, user data, and private API endpoints.
- Portfolio Sites — Allow full crawling of public work samples and case studies. Control how AI represents your creative output.
- Documentation Sites — Allow API docs and guides for AI training. Set clear attribution requirements for technical content.
- News & Media — Require attribution for all AI use of reporting. Block subscription-only content while allowing public articles.
How We Compare
| Feature | theStacc | Dark Visitors | Firecrawl | llmstxt.org |
|---|---|---|---|---|
| Free tier | Unlimited | Limited | Limited | Free |
| Sign-up required | No | No | Yes (API) | No |
| Auto-crawl from URL | No | No | Yes | No |
| Sitemap parsing | No | No | Yes | No |
| robots.txt analysis | No | No | No | No |
| Structured format | Yes | Yes | Yes | Basic |
| Format checking | Yes | Yes | No | No |
| AI crawler list | 10 crawlers | Basic | No | No |
| Template presets | 5 templates | 3 | No | No |
| Site type detection | No — you select it | No | Yes | No |
| Custom paths | Yes | No | No | No |
| Download option | Yes | Yes | No | Copy only |
Frequently Asked Questions
llms.txt is an emerging standard that lets websites control how AI crawlers access their content. It tells AI systems like ChatGPT, Claude, and Perplexity which parts of your site they can crawl, which they should avoid, and whether you require attribution when they use your content. As AI search grows, having an llms.txt file helps protect your intellectual property while maintaining visibility.
You enter your domain, contact email, and tick the kinds of content your site has, then pick a template. From those answers the tool suggests a set of common site sections, lets you edit every path, and writes a correctly formatted llms.txt with Allow/Disallow rules for major AI crawlers including GPTBot, ClaudeBot, PerplexityBot, and Google-Extended. It runs entirely in your browser and makes no network requests, so it never visits your website — check the suggested paths against your real URL structure before you upload the file.
Ten user agents: GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended (Gemini), CCBot (Common Crawl), FacebookBot (Meta), ChatGPT-User, OAI-SearchBot, Applebot, and Bingbot. You can toggle each one on or off, and you can add any others by hand once you have copied the file.
robots.txt controls what search engine crawlers can access for indexing in Google, Bing, etc. llms.txt is specifically for AI systems — it controls how AI crawlers use your content for training and generating responses. A site might allow Google to index everything but block AI crawlers from using content for training.
Requiring attribution means AI systems must credit your website when using your content in responses. This is recommended for content creators, publishers, and news sites. E-commerce and SaaS sites may prefer optional attribution. The tool lets you toggle this setting.
Upload the generated file to your website root directory so it is accessible at yourdomain.com/llms.txt. Most CMS platforms (WordPress, Webflow, Shopify) allow file uploads. For custom sites, place the file in your public folder.
Yes, 100% free with no signup. Everything runs in your browser, so there is no daily limit and none of your details — including your email address — are sent anywhere.
We offer five templates: Allow All Crawlers (full access), Block All AI Crawlers (maximum protection), Allow Blog Content Only (protect private areas), Allow Public Pages Only (block admin/checkout), and Custom (full control).
Yes. Before generating your llms.txt, toggle individual crawlers on or off. The generated file lists exactly the user agents you left enabled, so you can see at a glance which crawlers have access.
Yes. After generating your llms.txt, try our AI Visibility Checker to see how visible your site is to ChatGPT, Perplexity, Claude, and Gemini. theStacc also offers AI search optimization tools.
Without llms.txt, AI crawlers follow your robots.txt rules and their own policies. You have no explicit control over how AI systems use your content. Having llms.txt gives you direct control and signals to AI systems how you want your content handled.
No. It never fetches your homepage, sitemap, or robots.txt — no request leaves the page. The site sections it proposes are the common paths for the content types you ticked (/blog/, /docs/, /admin/ and so on), which is why you should compare them with your real URL structure and edit the allow and block lists before uploading the file.
Free tools are great. Automation is better.
Generating llms.txt is free. Content Indexing keeps your pages discoverable by search and AI engines automatically.
Sign up for free →