Top SitesThunderbit: AI Web Scraper - Scrape any website in 2 clicks

Machine Readiness

Stored receipt and evidence

Overall

16

Readable

20

Callable

40

Commerce

0

Payment

0

Machine Access

Inspect the site's MCP endpoint

Open MCP explorer

DialtoneApp can scan the stored discovery files for this domain, try the MCP initialize handshake, and show the raw protocol transcript.

Purchase boundary

crawl only

Control boundary

token required

Payment rails

None

Payment providers

None

Payment methods

None

Payment protocols

None

Payment assets

None

Payment networks

None

Capabilities

None

Verified payment surface

No

Crypto only

No

Readable docs

robots, openapi

Products

0

Variants

0

Priced variants

0

Currencies

0

Offers

0

Priced offers

0

Priced actions

0

Samples

Offer samples

No stored offer samples.

Samples

Action samples

No stored action samples.

Samples

Product samples

No stored product samples.

Document

robots.txt

Open robots.txt
# *
User-agent: *
Allow: /
Disallow: /*?
Disallow: /auth/*
Disallow: /internal/*
Disallow: /product/*
Disallow: /products/*
Disallow: /extension/*
Disallow: /pricing/subscription
Disallow: /pricing/next
Disallow: /chrome-extension
Disallow: /support
Disallow: /onboarding
Disallow: /playground
Disallow: /uploads
Disallow: /login
Disallow: /blog/all-articles/*
Disallow: /*/blog/all-articles/*
Disallow: /byte/*

# Allow Google Ads and Bing Ads crawlers to access /byte/
User-agent: AdsBot-Google
Allow: /byte/*

User-agent: AdsBot-Google-Mobile
Allow: /byte/*

User-agent: adidxbot
Allow: /byte/*

# Host
Host: https://thunderbit.com

# Sitemaps
Sitemap: https://thunderbit.com/sitemap.xml

Document

llms.txt

Not stored for this site.

Document

llms-full.txt

Not stored for this site.