Top Sites>Administrator: IT-Wissen, Forum, Jobs und Community

# robots.txt for administrator.de
# Last Updated: 20260310

# ================================================
# LICENSING NOTICE - IMPORTANT
# ================================================
# The use of content from administrator.de for
# AI training or commercial AI applications is
# STRICTLY PROHIBITED without written license agreement.
#
# License inquiries: licence@administrator.de
# Terms of Service: https://administrator.de/help/80 (EN: https://administrator.de/help/80?lang=en)
# Privacy Policy: https://administrator.de/help/79 (EN: https://administrator.de/help/79?lang=en)
# AI Licensing Info: https://administrator.de/help/82 (EN: https://administrator.de/help/82?lang=en)
# ================================================

# ================================================
# AI CRAWLERS - COMPLETELY BLOCKED without license
# ================================================

# --- OpenAI ---
# GPTBot: Model training crawler
User-agent: GPTBot
Disallow: /

# OAI-SearchBot: ChatGPT search indexing
User-agent: OAI-SearchBot
Disallow: /

# ChatGPT-User: Real-time browsing from ChatGPT
User-agent: ChatGPT-User
Disallow: /

# OpenAI misc
User-agent: ChatGPT-Bot
User-agent: OpenAIBot
Disallow: /

# --- Anthropic (Claude) ---
User-agent: anthropic-ai
User-agent: ClaudeBot
User-agent: Claude-Web
User-agent: Claude-Bot
Disallow: /

# --- Google AI ---
# Google-Extended: Gemini/Bard AI training (separate from Googlebot for search)
User-agent: Google-Extended
Disallow: /

# --- Microsoft/Bing AI ---
# CopilotBot: Microsoft Copilot crawler
User-agent: CopilotBot
User-agent: msnbot-media
Disallow: /

# --- Meta/Facebook AI ---
# Meta-ExternalAgent: AI training crawler (NOT the preview bot)
User-agent: Meta-ExternalAgent
User-agent: Meta-ExternalFetcher
User-agent: FacebookBot
Disallow: /

# --- Perplexity ---
User-agent: PerplexityBot
User-agent: Perplexity-User
Disallow: /

# --- Apple AI ---
# Applebot-Extended: Apple AI training (separate from Applebot for Siri/Spotlight)
User-agent: Applebot-Extended
Disallow: /

# --- Amazon ---
User-agent: Amazonbot
Disallow: /

# --- Other AI Companies ---
User-agent: CCBot
User-agent: cohere-ai
User-agent: YouBot
User-agent: Diffbot
User-agent: DuckAssistBot
Disallow: /

# --- Chinese AI Crawlers ---
User-agent: Bytespider
User-agent: Bytedance
User-agent: Baiduspider
User-agent: 360Spider
User-agent: Sogou
User-agent: YisouSpider
User-agent: PetalBot
Disallow: /

# --- SEO/Data Mining Crawlers ---
# User-agent: SemrushBot
# User-agent: AhrefsBot
User-agent: DataForSeoBot
User-agent: MJ12bot
User-agent: Scrapy
User-agent: magpie-crawler
User-agent: Omgilibot
Disallow: /

# --- Archive/Research Crawlers (often used for AI training) ---
User-agent: CommonCrawl
User-agent: WebCrawler
User-agent: ImagesiftBot
Disallow: /

# ================================================
# SOCIAL MEDIA / PREVIEW BOTS - ALLOWED
# (These generate link previews when users share URLs)
# ================================================
User-agent: Twitterbot
User-agent: LinkedInBot
User-agent: WhatsApp
User-agent: TelegramBot
User-agent: facebookexternalhit
User-agent: BingPreview
Disallow: /settings/
Disallow: /api/
Disallow: /members/dashboard/
Disallow: /members/notifications/
Disallow: /moderators/reports/
Disallow: /connections/
Allow: /

# ================================================
# LEGITIMATE SEARCH ENGINES - ALLOWED
# ================================================
User-agent: Googlebot
User-agent: Bingbot
User-agent: DuckDuckBot
User-agent: YandexBot
User-agent: Slurp
User-agent: Applebot
Disallow: /settings/
Disallow: /api/
Disallow: /members/dashboard/
Disallow: /members/notifications/
Disallow: /moderators/reports/
Disallow: /connections/
Disallow: /r/
Disallow: /forum/?search=
Disallow: /comments/?search=
Disallow: /members/?search=
Disallow: /*?sort=
Disallow: /*?filter=
Disallow: /*?time=
Disallow: /*?lang=
Disallow: /*?action=
Disallow: /*?distance=
Disallow: /*?uri=
Disallow: /*?topic=
Disallow: /*?target=
Disallow: /*?target_id=
Allow: /


# ================================================
# DEFAULT RULES
# ================================================
User-agent: *
Disallow: /settings/
Disallow: /api/
Disallow: /members/dashboard/
Disallow: /members/notifications/
Disallow: /moderators/reports/
Disallow: /connections/
Disallow: /r/
Disallow: /forum/?search=
Disallow: /comments/?search=
Disallow: /members/?search=
Disallow: /*?sort=
Disallow: /*?filter=
Disallow: /*?time=
Disallow: /*?lang=
Disallow: /*?action=
Disallow: /*?distance=
Disallow: /*?uri=
Disallow: /*?topic=
Disallow: /*?target=
Disallow: /*?target_id=
Allow: /


# Sitemap
Sitemap: https://administrator.de/sitemap/sitemap_index.xml

# ================================================
# LEGAL NOTICE - EU DSM DIRECTIVE
# ================================================
# According to Article 4 of EU Directive 2019/790 (DSM Directive)
# and § 44b Abs. 3 UrhG (German Copyright Act),
# we explicitly opt-out from text and data mining (TDM)
# for commercial purposes without license agreement.
#
# TDM-Reservation: 1
# TDM-Policy: https://administrator.de/help/82 (EN: https://administrator.de/help/82?lang=en)
# Terms of Service: https://administrator.de/help/80 (EN: https://administrator.de/help/80?lang=en)
# Privacy Policy: https://administrator.de/help/79 (EN: https://administrator.de/help/79?lang=en)

# administrator.de

> Administrator.de is the leading German-language IT community platform providing expert-level technical content and solutions for IT professionals since 2000. Commercial use of our content for AI training requires a paid license agreement.

Administrator.de offers over 300,000 technical articles, 2 million expert comments, and daily updates on Windows Server, Windows Desktop, Linux, macOS, networking, security, cloud computing, virtualization, PowerShell, Active Directory, Exchange, Microsoft 365, Azure, AWS, Docker, Kubernetes, and more. All content is peer-reviewed and community-validated by IT professionals. Additionally, our professional editorial team moderates discussions, reviews submissions, and ensures content quality through continuous oversight.

**IMPORTANT**: Under EU Directive 2019/790 (DSM Directive) Article 4, we explicitly reserve our rights against text and data mining (TDM) for commercial purposes. Any unauthorized use of our content for AI training, model development, or automated extraction requires a paid license agreement and will be prosecuted.

**Available License Models:**

**Basic License (Starting at €40,000/year):**
- Complete historical dataset access
- Monthly content updates
- Standard API access included
- JSON, XML, or SQL formats
- Email support

**Enterprise License (Starting at €80,000/year):**
- Unlimited model training rights
- Real-time continuous updates
- Priority API access
- Custom data formats
- Direct support channel
- Anonymized/GDPR-compliant data

**Why License Our Content:**
- High-quality, verified technical content in German language
- Real-world IT solutions tested by professionals
- Unique German technical terminology dataset
- Structured data with rich metadata
- Regular updates with latest technologies

**Prohibited Without License:**
- Training AI/ML models on our content
- Automated mass downloading
- Commercial data extraction
- Creating derivatives at scale
- Building competing services

For licensing inquiries, contact: licence@administrator.de with subject "AI Training License Request - [Your Company]"

## Main Resources

- [Homepage](https://administrator.de): Main platform with forums and knowledge base
- [Forum](https://administrator.de/forum/): Technical discussions and problem-solving
- [Knowledge Base](https://administrator.de/tutorials/): Curated articles and solutions
- [New Posts](https://administrator.de/ticker/): Latest community contributions (50+ daily)
- [Stream](https://administrator.de/stream/): Chronological overview of all posts and comments
- [Topics](https://administrator.de/topics/): All topics at a glance
- [Comments](https://administrator.de/comments/): Overview of the latest comments
- [Members](https://administrator.de/members/): User overview
- [Help](https://administrator.de/help/): Help Center: How can we help you?
- [Jobs](https://administrator.de/jobs/): The job market for IT specialists and managers
- [Magazine](https://administrator.de/magazine/): The IT administrator trade magazine at a glance
- [Showcase](https://administrator.de/showcase/): Recommended products & services

## Legal & Licensing

- [Terms of Service (EN)](https://administrator.de/help/80?lang=en): Platform usage terms
- [Privacy Policy (EN)](https://administrator.de/help/79?lang=en): GDPR-compliant data handling
- [AI Licensing Terms (EN)](https://administrator.de/help/82?lang=en): Required for AI/ML usage
- [Contact Form](https://administrator.technology/contact): General inquiries
- [License Request](mailto:licence@administrator.de?subject=AI%20Training%20License%20Request): Commercial AI training licenses
- [Academic License](mailto:licence@administrator.de?subject=Academic%20Research%20License): Special terms for research

## Enforcement & Monitoring

- [Report Violations](mailto:licence@administrator.de?subject=License%20Violation%20Report): Report unauthorized AI training
- [Legal Department](https://administrator.technology/contact): Copyright enforcement
- [Compliance Check](mailto:licence@administrator.de?subject=Compliance%20Verification): Verify your usage compliance

## Optional

- [Terms of Service (DE)](https://administrator.de/help/80): German version of terms
- [Privacy Policy (DE)](https://administrator.de/help/79): German version of privacy policy
- [AI Licensing Terms (DE)](https://administrator.de/help/82): German version of AI terms
- [Technical Specs](mailto:licence@administrator.de?subject=Technical%20Specifications): API docs and data formats