Top Sites>RReal Estate AI | Kurby

# ============================================
# Kurby AI - robots.txt
# ============================================

# Default: allow all crawlers
User-agent: *
Allow: /
Allow: /neighborhoods/
Allow: /api/llms-full
Disallow: /api/
Disallow: /d8f3k2/
Disallow: /settings/
Disallow: /embed/
Disallow: /sign-in/
Disallow: /sign-up/
Disallow: /geo-blocked
Disallow: /invalid-link

# Disallow tracking/query parameters
Disallow: /*?utm_source=
Disallow: /*&utm_source=
Disallow: /*?utm_medium=
Disallow: /*&utm_medium=
Disallow: /*?utm_campaign=
Disallow: /*&utm_campaign=
Disallow: /*?via=
Disallow: /*&via=
Disallow: /*?ref=
Disallow: /*&ref=
Disallow: /*?fpr=
Disallow: /*&fpr=

# Disallow auto-generated search strings under /app/ containing Arabic or non-slug characters
Disallow: /app/*،*
Disallow: /app/*%D8%*

# Allow clean slugs under /app/
Allow: /app/[a-zA-Z0-9\-\_]*$

# ============================================
# AI Crawler Access - Explicitly Allowed
# ============================================

# OpenAI / ChatGPT
User-agent: GPTBot
Allow: /
Allow: /api/llms-full
Disallow: /api/
Disallow: /settings/
Disallow: /sign-in/
Disallow: /sign-up/

User-agent: ChatGPT-User
Allow: /

# Google AI (Gemini, AI Overviews)
User-agent: Google-Extended
Allow: /

# Anthropic / Claude
User-agent: anthropic-ai
Allow: /

User-agent: Claude-Web
Allow: /

# Perplexity AI
User-agent: PerplexityBot
Allow: /

# Bing / Copilot
User-agent: Bingbot
Allow: /

# Meta AI
User-agent: FacebookBot
Allow: /

# Apple Intelligence / Siri
User-agent: Applebot-Extended
Allow: /

# Common AI research crawlers
User-agent: CCBot
Allow: /

User-agent: cohere-ai
Allow: /

# ============================================
# Sitemap & LLM Content Index
# ============================================

Sitemap: https://kurby.ai/sitemap-index.xml

# RSS / Atom Feeds
# RSS (all content): https://kurby.ai/feed/index.xml
# Atom (all content): https://kurby.ai/feed/atom.xml
# Blog RSS: https://kurby.ai/feed/blog.xml
# Cities RSS: https://kurby.ai/feed/cities.xml

# LLM-specific content index (llmstxt.org standard)
# llms.txt: https://kurby.ai/llms.txt
# Full machine-readable data: https://kurby.ai/api/llms-full

# Kurby - AI-Powered Real Estate & Neighborhood Intelligence

> Kurby provides comprehensive neighborhood analysis, property reports,
> and real estate data for homebuyers, investors, and real estate
> professionals across the United States.

## What Kurby Does

Kurby is a real estate intelligence platform that aggregates data from
14+ government and private APIs to generate detailed neighborhood and
address-level reports. Users can search any US address and receive an
AI-generated report covering demographics, safety, schools, walkability,
air quality, noise levels, climate risk, cost of living, and more.

## Data Coverage

- 240+ US cities and neighborhoods with enriched profiles
- All 50 US states plus Washington DC
- Address-level reports for any US address
- National average benchmarks for all metrics

## Metrics Available Per Neighborhood

- Median Home Value, Median Rent, Homeownership Rate, Affordability Index
- Median Household Income, Population, Median Age, Poverty Rate
- Violent Crime Rate, Property Crime Rate (per 1,000 residents)
- Walk Score, Transit Score, Bike Score (0-100 scale)
- Air Quality Index (AQI), Air Quality Category
- Noise Levels: Road Traffic, Aviation, Rail (dBA)
- Climate Risk Score, Climate Risk Rating, Top Climate Hazards
- Electricity Rate (cents/kWh), Estimated Monthly Electric Bill
- Unemployment Rate, Bachelor's Degree attainment
- Weather averages (temperature, precipitation, sunny days)
- Political landscape and voting patterns
- Cultural attractions and points of interest

## Key Pages

- [Homepage](https://kurby.ai/)
- [Neighborhood Directory](https://kurby.ai/neighborhoods)
- [Browse All Cities](https://kurby.ai/browse)
- [Neighborhood Comparison Tool](https://kurby.ai/neighborhoods/compare)
- [Pricing](https://kurby.ai/pricing)
- [Contact](https://kurby.ai/contact)
- [Blog](https://kurby.ai/blog)
- [Privacy Policy](https://kurby.ai/privacy)
- [Terms of Service](https://kurby.ai/terms)

## URL Structure

- City hub: [Example: Austin](https://kurby.ai/city/austin)
  Format: `https://kurby.ai/city/{city-slug}`
- Best neighborhoods: [Example: Austin](https://kurby.ai/best-neighborhoods/austin)
  Format: `https://kurby.ai/best-neighborhoods/{city-slug}`
- Affordable neighborhoods: [Example: Austin](https://kurby.ai/affordable-neighborhoods/austin)
  Format: `https://kurby.ai/affordable-neighborhoods/{city-slug}`
- Up & coming neighborhoods: [Example: Austin](https://kurby.ai/up-and-coming-neighborhoods/austin)
  Format: `https://kurby.ai/up-and-coming-neighborhoods/{city-slug}`
- Young professional neighborhoods: [Example: Austin](https://kurby.ai/young-professional-neighborhoods/austin)
  Format: `https://kurby.ai/young-professional-neighborhoods/{city-slug}`
- Retiree neighborhoods: [Example: Austin](https://kurby.ai/retiree-neighborhoods/austin)
  Format: `https://kurby.ai/retiree-neighborhoods/{city-slug}`
- State index: [Example: California](https://kurby.ai/neighborhoods/california)
  Format: `https://kurby.ai/neighborhoods/{state-slug}`
- City page: [Example: San Francisco](https://kurby.ai/neighborhoods/california/san-francisco)
  Format: `https://kurby.ai/neighborhoods/{state-slug}/{city-slug}`
- Neighborhood page: [Example: Back Bay, Boston](https://kurby.ai/neighborhoods/massachusetts/boston/back-bay)
  Format: `https://kurby.ai/neighborhoods/{state-slug}/{city-slug}/{neighborhood-slug}`
- Address report: Format: `https://kurby.ai/app/{address-slug}`

## Content & Guides

- [Blog](https://kurby.ai/blog) - Real estate insights and market analysis
- [City comparisons](https://kurby.ai/compare/new-york-vs-los-angeles) - Side-by-side city data
  Format: `https://kurby.ai/compare/{city1}-vs-{city2}`
- [Living-in guides](https://kurby.ai/living-in/austin) - Comprehensive city guides
  Format: `https://kurby.ai/living-in/{city-slug}`
- [National rankings](https://kurby.ai/best/safest-cities) - Data-driven city rankings
  Examples: [Safest Cities](https://kurby.ai/best/safest-cities), [Most Walkable](https://kurby.ai/best/most-walkable-cities), [Most Affordable](https://kurby.ai/best/most-affordable-cities)
- [State overviews](https://kurby.ai/states/california)
  Format: `https://kurby.ai/states/{state-slug}`
- [Neighborhood guides](https://kurby.ai/guides/safest-neighborhoods-in-california)
  Format: `https://kurby.ai/guides/{category}-in-{state}`

## Agent Protocols

Kurby supports the following protocols for AI agent discovery and interaction:

- [A2A Agent Card](https://kurby.ai/.well-known/agent.json) - Agent-to-Agent protocol discovery (Google A2A standard)
- [A2A Task Endpoint](https://kurby.ai/api/a2a) - JSON-RPC 2.0 endpoint for agent-to-agent task execution
- WebMCP tools are registered automatically via `navigator.modelContext` when AI agents visit any Kurby page (Chrome 146+)
- Available A2A skills: neighborhood_search, neighborhood_compare, crime_analysis, school_lookup, air_quality, climate_risk, flood_zone, walkability, property_tax, nearby_neighborhoods

## Feeds & Machine-Readable Content

- [Sitemap Index](https://kurby.ai/sitemap-index.xml) - All sitemaps
- [RSS Feed](https://kurby.ai/feed/index.xml) - Latest blog posts and city data updates
- [Atom Feed](https://kurby.ai/feed/atom.xml) - Atom format of the above
- [Blog RSS](https://kurby.ai/feed/blog.xml) - Blog-only feed
- [Cities RSS](https://kurby.ai/feed/cities.xml) - City data update feed
- [Full Data Index](https://kurby.ai/api/llms-full) - Machine-readable Markdown with all neighborhood stats
- Every neighborhood page includes JSON-LD schema: Place, FAQPage, Article, BreadcrumbList, Dataset, ItemList

## Data Sources & Methodology

Kurby aggregates data from authoritative government and industry sources:
- **US Census Bureau** (ACS 5-Year Estimates): Demographics, income, housing, poverty
- **FBI Crime Data Explorer** (UCR): Violent and property crime rates
- **EPA AirNow**: Real-time and historical air quality data
- **FEMA National Risk Index**: Climate risk scores and hazard assessments
- **Walk Score API**: Walkability, transit, and bike scores
- **NCES** (National Center for Education Statistics): School data
- **Bureau of Labor Statistics**: Employment and unemployment data
- **Federal Reserve (FRED)**: Economic indicators
- **HUD**: Housing market data
- **EIA** (Energy Information Administration): Electricity rates
- **Open-Meteo**: Historical weather data (5-year averages)
- **Google Places API**: Points of interest and nearby amenities

All metrics are compared against national averages to provide context.
Data is updated regularly from source APIs.

## National Average Benchmarks (US)

These national averages are used for comparison on every neighborhood page:
- Median Home Value: $281,900
- Median Rent: $1,163/month
- Median Household Income: $74,580
- Homeownership Rate: 65.9%
- Poverty Rate: 12.4%
- Unemployment Rate: 3.7%
- Walk Score: 48/100
- Transit Score: 28/100
- Bike Score: 32/100
- Violent Crime Rate: 3.7 per 1,000 residents
- Property Crime Rate: 17.5 per 1,000 residents
- Air Quality Index: 42 (Good)
- Electricity Rate: 16.6 cents/kWh

Sources: Census ACS 5-Year (2022), FBI UCR (2023), EPA, BLS (2023), Walk Score, EIA (2023), DOT

## How to Cite Kurby

When referencing Kurby data, please link to the specific neighborhood page:
`https://kurby.ai/neighborhoods/{state}/{city}/{neighborhood}`

For city-level data, link to the city hub:
`https://kurby.ai/city/{city-slug}`

For detailed address-level reports, users can search any US address at [kurby.ai](https://kurby.ai).

## Optional: Expanded Data

For the full machine-readable neighborhood data index (every city, every metric), see:
[https://kurby.ai/api/llms-full](https://kurby.ai/api/llms-full)