Machine Readiness
Stored receipt and evidence
30
100
0
0
0
Samples
No stored offer samples.
Samples
No stored action samples.
Samples
No stored product samples.
Document
# ============================================================================= # canna-pet.com — robots.txt # Updated: 2026-04-13 # Comprehensive crawl directives with AI crawler management # ============================================================================= # ─── Default: All crawlers ─────────────────────────────────────────────────── User-agent: * Allow: / # WordPress admin & internals Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-json/ # WooCommerce — defensive (checkout happens on blackdahlia.co, # but block these in case WC generates the URLs) Disallow: /checkout/ Disallow: /cart/ Disallow: /my-account/ Disallow: /order-received/ Disallow: /add-to-cart/ Disallow: /*add-to-cart=* Disallow: /*add_to_wishlist=* # Search & filtered views (duplicate/thin content) Disallow: /?s= Disallow: /*?s=* Disallow: /*?orderby=* Disallow: /*?filter* Disallow: /*?product_cat=* # Paginated archives deep pages Disallow: /page/ # Feed URLs Disallow: /feed/ Disallow: /comments/feed/ Disallow: /*/feed/ # Author archives (thin pages) Disallow: /author/ # WordPress login / registration Disallow: /wp-login.php Disallow: /wp-register.php # Allow LLMs.txt endpoints Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full # Allow key content paths explicitly Allow: /product/ Allow: /products/ Allow: /breed/ Allow: /faq/ Allow: /blog/ Allow: /health-concerns/ # ─── AI Crawlers — Block training scraping ─────────────────────────────────── # These crawlers scrape for LLM training data. Block them from bulk crawling # while still serving /llms.txt and /llms-full.txt for structured discovery. User-agent: ClaudeBot Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: GPTBot Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: PerplexityBot Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: Google-Extended Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: meta-externalagent Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: Applebot-Extended Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: CCBot Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: anthropic-ai Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / User-agent: cohere-ai Allow: /llms.txt Allow: /llms-full.txt Allow: /llms Allow: /llms-full Disallow: / # ─── Sitemap ───────────────────────────────────────────────────────────────── Sitemap: https://canna-pet.com/sitemap_index.xml
Document
# Canna-Pet® — canna-pet.com > Premium pet wellness supplements — organic hemp CBD products for dogs and cats. > Last updated: 2026-04-26 ## Brand: Canna-Pet® Organic hemp CBD wellness supplements for dogs and cats. Biscuits, capsules, and MaxCBD liquid formulas dosed by pet weight. 0% THC (non-detectable). - Products: 25 SKUs - Categories: Dogs, cats, Canna Pet Max, biscuit packs, advanced biscuits, advanced capsules, MaxCBD liquid, packages - Catalog: https://canna-pet.com/products/ - Products are available for purchase at https://blackdahlia.co ## Content resources - Breed guides: 198 breed profiles - FAQ: 34 frequently asked questions - Health concerns: 519 condition pages - Blog: https://canna-pet.com/blog/ ## Machine-readable catalog Full catalog text: https://canna-pet.com/llms-full.txt Product feed (JSON): https://canna-pet.com/wp-json/wc/v3/products?status=publish&per_page=100 Product categories: https://canna-pet.com/wp-json/wc/v3/products/categories?per_page=50 ## Policies - Returns: https://blackdahlia.co/return-policy/ - Shipping: https://canna-pet.com/shipping-info/ - Privacy: https://canna-pet.com/privacy-policy/ - Contact: https://canna-pet.com/contact/ ## Crawler permissions AI crawlers may index all public product, category, breed, FAQ, and blog pages. Restricted: /wp-admin/, /checkout/, /cart/, /my-account/
Document
Not stored for this site.