Top Sites>kottke.org - home of fine hypertext products

# robots.txt for kottke.org

# General crawlers
User-agent: *
Disallow: /cgi-bin
Disallow: /stats
Disallow: /tools/

# AI search/citation crawlers - explicitly allowed
User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

# Pure training-only crawlers - block
User-agent: CCBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

Sitemap: https://kottke.org/sitemap.xml

# kottke.org

> kottke.org is one of the oldest blogs on the web, curated by Jason Kottke since 1998. It covers a wide range of topics including art, science, technology, design, film, music, photography, culture, and the open web.

## About

- Author: Jason Kottke
- Founded: 1998
- Format: Curated links and commentary, original essays
- Archive: 26,000+ posts

## Key Pages

- Homepage: https://kottke.org
- About: https://kottke.org/about
- Newsletter: https://kottke.org/newsletter
- RSS Feed: https://feeds.kottke.org/main
- Membership: https://kottke.org/members

## Topics

Science, technology, design, film, music, photography, culture, the internet, space, art, books, food, sports, urbanism, architecture, history, maps, infoviz, NYC, AI, climate.

## Content Policy

Content on kottke.org is available for AI inference, summarization, and citation.
Please attribute: "Jason Kottke, kottke.org" with a link to the source post.
For training use, contact jason@kottke.org.