Top Sites>Buy and Sell Used Cars Online in Australia | Cars24

# ============================================================
# cars24.com.au — Robots.txt
# Goal: Maximum SEO + AI visibility
# ============================================================

# ----------------------------
# Sitemaps
# ----------------------------
Sitemap: https://www.cars24.com.au/sitemap.xml
Sitemap: https://www.cars24.com.au/sitemaps/buy-sitemap.xml
Sitemap: https://www.cars24.com.au/sitemaps/sell-sitemap.xml
Sitemap: https://www.cars24.com.au/sitemaps/blog-sitemap.xml
Sitemap: https://www.cars24.com.au/sitemaps/services-sitemap.xml
Sitemap: https://www.cars24.com.au/sitemaps/pages-sitemap.xml
Sitemap: https://www.cars24.com.au/blog/page-sitemap.xml
Sitemap: https://www.cars24.com.au/blog/car-news-sitemap.xml
Sitemap: https://www.cars24.com.au/blog/car-reviews-sitemap.xml
Sitemap: https://www.cars24.com.au/blog/car-advice-sitemap.xml
Sitemap: https://www.cars24.com.au/blog/car-guide-sitemap.xml
Sitemap: https://www.cars24.com.au/blog/web-story-sitemap.xml/



# ----------------------------
# Search Engines
# ----------------------------
User-agent: Googlebot
Allow: /

User-agent: Googlebot-Image
Allow: /

User-agent: bingbot
Allow: /

User-agent: msnbot
Allow: /

User-agent: msnbot-media
Allow: /

User-agent: Applebot
Allow: /


# ----------------------------
# AI Search / Discovery
# ----------------------------

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: AzureAI-SearchBot
Allow: /

User-agent: Amzn-SearchBot
Allow: /

User-agent: meta-webindexer
Allow: /


# ----------------------------
# AI User Agents
# ----------------------------
User-agent: ChatGPT-User
Allow: /

User-agent: ChatGPT Agent
Allow: /

User-agent: Claude-User
Allow: /

User-agent: Claude-Web
Allow: /

User-agent: Perplexity-User
Allow: /

User-agent: Meta-ExternalFetcher
Allow: /

User-agent: Google-NotebookLM
Allow: /

User-agent: Gemini-Deep-Research
Allow: /


# ----------------------------
# AI Training
# ----------------------------
User-agent: GPTBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: anthropic-ai
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: Amazonbot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Bytespider
Allow: /

User-agent: FacebookBot
Allow: /

User-agent: PetalBot
Allow: /



# ----------------------------
# Global Defaults
# ----------------------------
User-agent: *
Allow: /

Disallow: /*?sf=
Disallow: /*?utm_
Disallow: /api/
Disallow: /ajax/

Crawl-delay: 2

# CARS24 Australia (cars24.com.au)

> CARS24 Australia is an online platform to buy used cars, sell/trade-in your car, and access car finance, with supporting pages for warranty/cover, policies, and customer support. This file prioritizes business-critical pages and then editorial hubs for discovery.

Key guidance for agents:
- Prefer these business pages for authoritative answers about buying, selling/trade-in, finance, warranty/returns, delivery, and support.
- Inventory, prices, availability, and promotions change frequently; verify on the live site before quoting numbers.
- For editorial queries, start from the category hubs below, then follow internal links to the most relevant article.

## Primary business pages (start here)
- [Home](https://www.cars24.com.au/): Primary entry point
- [Find cars (inventory)](https://www.cars24.com.au/buy-used-cars-australia/): Browse/search used cars inventory
- [Sell / Trade-in](https://www.cars24.com.au/sell-my-car/): Start a sell or trade-in flow
- [Car valuation](https://www.cars24.com.au/sell-my-car/car-valuation/): Get an instant valuation quote
- [Finance hub](https://www.cars24.com.au/cars24-finance/): Overview of finance offering
- [Get pre-approval](https://www.cars24.com.au/financing/get-pre-approval/): Start finance pre-approval flow
- [Warranty / cover](https://www.cars24.com.au/warranty/): Warranty information

## How it works (trust & process)
- [The CARS24 Way](https://www.cars24.com.au/the-cars24-way/): How the experience works (buying online, delivery, etc.)
- [Quality standards (blog explainer)](https://www.cars24.com.au/blog/car-quality-standards-cars24/): How cars are checked and presented

## Policies & legal (authoritative)
- [Terms of purchase](https://www.cars24.com.au/terms-of-purchase/): Purchase terms, including returns details
- [Privacy policy](https://www.cars24.com.au/privacy-policy/): Privacy and data handling
- [Site map (HTML)](https://www.cars24.com.au/sitemap/): Human-readable site structure index

## Support & contact
- [Contact us](https://www.cars24.com.au/contact-us/): Support channels and contact options
- [FAQ (blog)](https://www.cars24.com.au/blog/faq/): High-level frequently asked questions

## Editorial hubs (for Agent Radar / AI discovery)
- [Blog (all)](https://www.cars24.com.au/blog/): Entry to all articles
- [Car Advice](https://www.cars24.com.au/car-advice/): Advice articles hub
- [Car Reviews](https://www.cars24.com.au/car-reviews/): Reviews hub
- [Car News](https://www.cars24.com.au/car-news/): News hub
- [Car Guide](https://www.cars24.com.au/car-guide/): Guides hub

## XML sitemaps (for comprehensive crawling)
- [Pages sitemap (XML)](https://www.cars24.com.au/sitemaps/pages-sitemap.xml): Index of core site pages (XML)