Machine Readiness
Stored receipt and evidence
18
20
0
50
0
Samples
No stored offer samples.
Samples
No stored action samples.
Samples
Focus Stack
Metabolism Stack
Detox Stack
Document
# ROBOTS.TXT by Visibilidadon # —————————————————————————————— # Sitemap locations # —————————————————————————————— Sitemap: https://blackforestsupplements.com/sitemap.xml Sitemap: https://blackforestsupplements.com/sitemap_blogs_1.xml # —————————————————————————————— # Global rules # —————————————————————————————— User-agent: * # Allow all static assets so Google can render your pages correctly Allow: /assets/ Allow: /cdn/ Allow: /*.css$ Allow: /*.js$ Allow: /*.png$ Allow: /*.jpg$ Allow: /*.jpeg$ Allow: /*.svg$ Allow: /*.woff$ Allow: /*.woff2$ # —————————————————————————————— # Block only truly private or checkout/account areas # —————————————————————————————— Disallow: /admin Disallow: /account Disallow: /cart Allow: /checkouts/internal/preloads.js Disallow: /orders Disallow: /carts Disallow: /services/login_with_shop # Keep preview URLs and one-off Shopify endpoints out of crawl Disallow: /*?*oseid=* Disallow: /*preview_theme_id* Disallow: /*preview_script_id* # —————————————————————————————— # Specialized bots # —————————————————————————————— # AdsBot can’t see carts or checkout, same as above User-agent: adsbot-google Disallow: /cart Disallow: /carts Disallow: /orders Disallow: /services/login_with_shop # Block overly aggressive crawlers entirely or slow them down User-agent: Nutch Disallow: / User-agent: AhrefsBot Crawl-delay: 10 Disallow: / User-agent: AhrefsSiteAudit Crawl-delay: 10 Disallow: / User-agent: MJ12bot Crawl-delay: 10 User-agent: Pinterest Crawl-delay: 1 # —————————————————————————————— # AI Search Bots (NEW SECTION) # —————————————————————————————— # Allow Google's AI models access to cart and account pages. User-agent: Google-Extended Allow: /account/ Allow: /cart/ # Allow OpenAI's models access to cart and account pages. User-agent: ChatGPT-User Allow: /account/ Allow: /cart/ # Allow Perplexity's models access to cart and account pages. User-agent: PerplexityBot Allow: /account/ Allow: /cart/ User-agent: OAI-SearchBot Allow: /account/ Allow: /cart/ # Allow OpenAI's models access to cart and account pages. User-agent: Googlebot Allow: /account/ Allow: /cart/ # Allow Perplexity's models access to cart and account pages. User-agent: Perplexity-User Allow: /account/ Allow: /cart/ # Allow Perplexity's models access to cart and account pages. User-agent: Claude-User Allow: /account/ Allow: /cart/ # Allow Perplexity's models access to cart and account pages. User-agent: Claude-SearchBot Allow: /account/ Allow: /cart/
Document
Not stored for this site.
Document
Not stored for this site.