# Allow key AI / LLM crawlers User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Google-Extended Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / # Allow Bing search + Bing AI / Copilot User-agent: Bingbot Allow: / User-agent: BingPreview Allow: / User-agent: MicrosoftEdge Allow: / # Allow major SEO tools User-agent: SemrushBot Allow: / User-agent: AhrefsBot Allow: / # Default rules for all other bots User-agent: * Disallow: /wp-admin/ Disallow: /tutor/ Disallow: /wp-content/uploads/wc-logs/ Disallow: /wp-content/uploads/woocommerce_transient_files/ Disallow: /wp-content/uploads/woocommerce_uploads/ Allow: /wp-admin/admin-ajax.php # Sitemap for the NEW domain Sitemap: https://yogafit.com/sitemap_index.xml