LLMs.txt Generator
How to Use the LLMs.txt Generator
The LLMs.txt Generator helps you create an llms.txt file for your website. This file provides clear information about your website, documentation, resources, and important pages so that AI systems can better understand and use your content.
Step 1: Enter Your Website Information
Start by adding your basic website details:
- Website Name — Enter the name of your website or brand.
- Website URL — Add your main website address.
- Description — Write a short summary explaining what your website provides.
Example:
A platform offering online tools, guides, and resources for developers and businesses.
Step 2: Add Important Website Links
Include links that help AI systems discover your important resources:
- Documentation URL — Add your documentation or knowledge base link.
- API URL — Add your API documentation if your website provides an API.
- Privacy Policy — Add your privacy policy page.
- Terms of Service — Add your terms and conditions page.
- Contact URL or Email — Provide a way for users and systems to find contact information.
- Sitemap URL — Add your website sitemap link.
Step 3: Configure Crawler Information
Use the crawler settings to provide additional instructions:
Allowed Crawlers
Add the AI crawlers or services you want to allow. Enter one item per line.
Example:
GPTBot
ClaudeBot
Google-Extended
Disallowed Paths
Add website sections that should not be included.
Example:
/admin
/private
/account
Step 4: Add Custom Information
The extra sections field allows you to add your own Markdown content.
You can include:
- Product information
- Important pages
- Developer resources
- FAQs
- Content guidelines
- Additional instructions
Example:
## Resources
- Tutorials: https://example.com/tutorials
- Blog: https://example.com/blog
Step 5: Generate Your llms.txt File
After completing the form:
- Click the Generate llms.txt button.
- Review the generated content.
- Make any changes if needed.
Your file will be displayed in the preview area.
Step 6: Copy or Download the File
Use the available options:
- Copy — Copy the generated content to your clipboard.
- Download — Save the file as
llms.txt.
Step 7: Upload llms.txt to Your Website
Upload the generated file to your website’s root directory.
Your file should be available at:
https://yourwebsite.com/llms.txt
Make sure the file is publicly accessible so AI systems and visitors can read it.
Best Practices
For the best results:
- Keep your description clear and accurate.
- Add only important and publicly available links.
- Update your
llms.txtfile when your website changes. - Avoid adding private or sensitive information.
- Use clear headings and simple language.
A well-structured llms.txt file helps AI systems understand your website structure and find your most useful content.