Top Sites>Jobs and Recruitment on Reed.co.uk, the UK's #1 job site

# Explicit Allow Rules for Major AI and LLM Crawlers

# OpenAI bots
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: GPTBot
Allow: /

# Anthropic (Claude)
User-agent: AnthropicBot
Allow: /

#Perplexity AI
User-agent: PerplexityBot
Allow: /

# allow the google advertising bot full access
User-agent: Mediapartners-Google
Allow: /

User-agent: TalkTalk Virus Alerts Scanning Engine
Disallow: /jobs/

User-agent: JoobleBot
Crawl-delay: 3

User-agent: AhrefsBot
Crawl-delay: 3

# Unified rules for all bots to avoid duplication
# Explicitly including Googlebot and Bingbot to ensure they follow the same rules as other crawlers
User-agent: *
User-agent: bingbot
User-agent: Googlebot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: GPTBot
User-agent: AnthropicBot
User-agent: PerplexityBot
Disallow: /linkclicked.html
Disallow: /api/
Disallow: /handlers/
Disallow: /salary-checker

Disallow: /*?sourceInternal=
Disallow: /*?directjob
Disallow: /*buy?path

# Courses
Disallow: /courses/api/
Disallow: /courses/errorlogging
Disallow: /courses/logging/logcourseview
Disallow: /courses/*/enquiry/externalredirect
Disallow: /courses/externalredirect/*
Disallow: /courses/account/signin?returnUrl=
Disallow: /courses/*?*pricerangeoptions=
Disallow: /courses/*?*sortby=
Disallow: /courses/*?*pagesize=
Disallow: /courses/*?*location=
Disallow: /courses/*?*pricefrom=
Disallow: /courses/*?*priceto=
Disallow: /courses/*?*duration=
Disallow: /courses/buy-now/
Disallow: /courses/*/buy-now
Disallow: /courses/on-demand/
Disallow: /courses/*/on-demand
Disallow: /courses/checkout/createdigitalwallet

# Sitemap URL:
Sitemap: https://www.reed.co.uk/sitemap.xml
Sitemap: https://www.reed.co.uk/priority-jobseeker-sitemap.xml
Sitemap: https://www.reed.co.uk/courses/sitemap.xml

#Allow: WebSurge

# llms.txt for Reed.co.uk
version: 1.0
site: https://www.reed.co.uk
purpose: Guidance for language models and automated agents
last updated: 2025-11-17

# Reed.co.uk
> Reed.co.uk is the UK's leading online recruitment platform, providing access to over 200,000 job listings from 19,000 recruiters, comprehensive CV management, expert career advice, and 155,000 professional courses from 1,400 UK learning providers, supporting career development and upskilling across multiple fields. This file provides guidance for language models and other automated agents.

## Core site overview
- [Home page](https://www.reed.co.uk): Main entry point to jobs, recruitment and courses.
- [About us](https://www.reed.co.uk/about): Background on Reed's services and mission.
- [Employer area](https://www.reed.co.uk/recruiter): Job posting and recruitment solutions for employers.


## Content & resources
- [Career advice articles](https://www.reed.co.uk/career-advice/): Articles on job search, career planning, CV writing, and interviews.
- [Career guides articles](https://www.reed.co.uk/courses/career-guides): Structured guides on professional development in various fields, outlining skills, qualifications, and pathways to advance careers.
- [Recruitment insights blog](https://www.reed.co.uk/recruiter-advice/): Blog posts, research and resources on the UK labour market, offering guidance on recruitment strategies, job posting, and hiring best practices.
- [Help & FAQs - Jobs](https://www.reed.co.uk/help): Information and guidance on job applications, account management, site features and common questions for job seekers.
- [Help & FAQs - Courses](https://www.reed.co.uk/courses/help): Information and guidance on course enrollment, learning resources, platform features and common questions for course students.

## Legal & policy
- [Privacy policy](https://www.reed.co.uk/policies): Information on data handling, privacy and terms and conditions.

## Deep-dive
- [Job listings search page](https://www.reed.co.uk/jobs): Comprehensive access to all job listings.
- [Recruiter Service Post a Job](https://www.reed.co.uk/recruiter/post-a-job): Recruiter – Post a Job: Platform for recruiters to create job postings.
- [Recruiter Service CV Search](https://www.reed.co.uk/recruiter/cv-search): Recruiter – CV Search: Tool for recruiters to search and identify qualified candidates.
- [Recruiter ATS Integration Service](https://www.reed.co.uk/recruiter/ats-integration): Recruiter – ATS Integration: Service for integrating Reed.co.uk with ATS to streamline hiring.
- [Reed.ai](https://reed.ai/): Reed.ai: AI-powered recruitment platform connecting employers with candidates for efficient job matching.
- [Courses listings search page](https://www.reed.co.uk/courses/): Courses listings search page: Comprehensive access to all course listings.

## Technical
- [Sitemap](https://www.reed.co.uk/sitemap.xml): Full index of site URLs for reference and crawling purposes.
- [Robots.txt](https://www.reed.co.uk/robots.txt): Rules controlling automated access and crawling of Reed.co.uk content.