LLMs.txt Generator

LLMs.txt Generator


How to Use the LLMs.txt Generator

The LLMs.txt Generator helps you create an llms.txt file for your website. This file provides clear information about your website, documentation, resources, and important pages so that AI systems can better understand and use your content.

Step 1: Enter Your Website Information

Start by adding your basic website details:

  • Website Name — Enter the name of your website or brand.
  • Website URL — Add your main website address.
  • Description — Write a short summary explaining what your website provides.

Example:

A platform offering online tools, guides, and resources for developers and businesses.

Step 2: Add Important Website Links

Include links that help AI systems discover your important resources:

  • Documentation URL — Add your documentation or knowledge base link.
  • API URL — Add your API documentation if your website provides an API.
  • Privacy Policy — Add your privacy policy page.
  • Terms of Service — Add your terms and conditions page.
  • Contact URL or Email — Provide a way for users and systems to find contact information.
  • Sitemap URL — Add your website sitemap link.

Step 3: Configure Crawler Information

Use the crawler settings to provide additional instructions:

Allowed Crawlers

Add the AI crawlers or services you want to allow. Enter one item per line.

Example:

GPTBot
ClaudeBot
Google-Extended

Disallowed Paths

Add website sections that should not be included.

Example:

/admin
/private
/account

Step 4: Add Custom Information

The extra sections field allows you to add your own Markdown content.

You can include:

  • Product information
  • Important pages
  • Developer resources
  • FAQs
  • Content guidelines
  • Additional instructions

Example:

## Resources

- Tutorials: https://example.com/tutorials
- Blog: https://example.com/blog

Step 5: Generate Your llms.txt File

After completing the form:

  1. Click the Generate llms.txt button.
  2. Review the generated content.
  3. Make any changes if needed.

Your file will be displayed in the preview area.

Step 6: Copy or Download the File

Use the available options:

  • Copy — Copy the generated content to your clipboard.
  • Download — Save the file as llms.txt.

Step 7: Upload llms.txt to Your Website

Upload the generated file to your website’s root directory.

Your file should be available at:

https://yourwebsite.com/llms.txt

Make sure the file is publicly accessible so AI systems and visitors can read it.

Best Practices

For the best results:

  • Keep your description clear and accurate.
  • Add only important and publicly available links.
  • Update your llms.txt file when your website changes.
  • Avoid adding private or sensitive information.
  • Use clear headings and simple language.

A well-structured llms.txt file helps AI systems understand your website structure and find your most useful content.

Add as Preferred Source