# robots.txt for xyz.frl Dynamic DNS service User-agent: * Allow: / Allow: /llms.txt Allow: /llms-full.txt Allow: /.well-known/mcp.json Allow: /generate Allow: /nic/update Allow: /mcp Allow: /use-cases Allow: /use-cases/ Disallow: /metrics # AI & LLM Agent Crawlers User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: CCBot Allow: / # Machine-Readable Agent Discovery # Model Context Protocol: https://xyz.frl/mcp/sse # MCP Server Manifest: https://xyz.frl/.well-known/mcp.json # LLM Context: https://xyz.frl/llms.txt # LLM Full Reference: https://xyz.frl/llms-full.txt