Machine Readiness
Stored receipt and evidence
20
65
0
0
0
Samples
No stored offer samples.
Samples
No stored action samples.
Samples
No stored product samples.
Document
# -------------------------------------------------- # Googlebot — respects only Disallow (use Search Console for crawl rate) # -------------------------------------------------- User-agent: Googlebot Disallow: /wp-admin/ Disallow: /tag/ Disallow: /blog/tag/ Disallow: /louisville-events/tag/ Disallow: /louisville-events/category/*/day/ Disallow: /louisville-events/category/*/month/ Disallow: /louisville-events/category/*/list/ Disallow: /*?fwp_article_tag= Disallow: /*?fwp_categories= Disallow: /*?posts_per_page= Disallow: /*?fwp_paged= Disallow: /*outlook-ical=* Disallow: /*ical=* Disallow: /*tribe-bar-date=* Disallow: /*eventDisplay=* Disallow: /search/ Disallow: /keywords/ Allow: /wp-admin/admin-ajax.php # -------------------------------------------------- # Bingbot — respects Crawl-delay # -------------------------------------------------- User-agent: bingbot Crawl-delay: 10 Disallow: /wp-admin/ Disallow: /tag/ Disallow: /blog/tag/ Disallow: /louisville-events/tag/ Disallow: /louisville-events/category/*/day/ Disallow: /louisville-events/category/*/month/ Disallow: /louisville-events/category/*/list/ Disallow: /*?fwp_article_tag= Disallow: /*?fwp_categories= Disallow: /*?posts_per_page= Disallow: /*?fwp_paged= Disallow: /*outlook-ical=* Disallow: /*ical=* Disallow: /*tribe-bar-date=* Disallow: /*eventDisplay=* Disallow: /search/ Disallow: /keywords/ Allow: /wp-admin/admin-ajax.php # -------------------------------------------------- # Applebot — Siri/Spotlight # -------------------------------------------------- User-agent: Applebot Crawl-delay: 15 Disallow: /wp-admin/ Disallow: /tag/ Disallow: /blog/tag/ Disallow: /louisville-events/tag/ Disallow: /louisville-events/category/*/day/ Disallow: /louisville-events/category/*/month/ Disallow: /louisville-events/category/*/list/ Disallow: /*?fwp_article_tag= Disallow: /*?fwp_categories= Disallow: /*?posts_per_page= Disallow: /*?fwp_paged= Disallow: /*outlook-ical=* Disallow: /*ical=* Disallow: /*tribe-bar-date=* Disallow: /*eventDisplay=* Disallow: /search/ Disallow: /keywords/ Allow: /wp-admin/admin-ajax.php # -------------------------------------------------- # Block non-essential SEO spy / scraper bots entirely # -------------------------------------------------- User-agent: PetalBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: Bytespider Disallow: / User-agent: YandexBot Disallow: / # -------------------------------------------------- # AI scrapers — block unless you want to be in their training data # -------------------------------------------------- User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: CCBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: FacebookBot Disallow: / # -------------------------------------------------- # TikTok — heavy crawler with no SEO value # -------------------------------------------------- User-agent: TikTokSpider Disallow: / # -------------------------------------------------- # Default — all other bots # -------------------------------------------------- User-agent: * Crawl-delay: 15 Disallow: /wp-admin/ Disallow: /tag/ Disallow: /blog/tag/ Disallow: /louisville-events/tag/ Disallow: /louisville-events/category/*/day/ Disallow: /louisville-events/category/*/month/ Disallow: /louisville-events/category/*/list/ Disallow: /*?fwp_article_tag= Disallow: /*?fwp_categories= Disallow: /*?posts_per_page= Disallow: /*?fwp_paged= Disallow: /*outlook-ical=* Disallow: /*ical=* Disallow: /*tribe-bar-date=* Disallow: /*eventDisplay=* Disallow: /search/ Disallow: /keywords/ Allow: /wp-admin/admin-ajax.php Sitemap: https://uoflhealth.org/sitemap_index.xml Sitemap: https://uoflhealth.org/post-sitemap.xml Sitemap: https://uoflhealth.org/page-sitemap.xml Sitemap: https://uoflhealth.org/providers-sitemap.xml Sitemap: https://uoflhealth.org/pss-sitemap.xml Sitemap: https://uoflhealth.org/locations-sitemap.xml Sitemap: https://uoflhealth.org/health-library-sitemap.xml Sitemap: https://uoflhealth.org/video-library-sitemap.xml Sitemap: https://uoflhealth.org/podcasts-sitemap.xml Sitemap: https://uoflhealth.org/category-sitemap.xml
Document
Not stored for this site.
Document
Not stored for this site.