User-agent: * Disallow: /result Disallow: /track Disallow: /api/ Disallow: /terms Disallow: /refund Disallow: /cookies Disallow: /privacy Disallow: /do-not-sell Disallow: /set-locale/ Disallow: /profile/organizations/claim/* Disallow: /*/profile/organizations/claim/* Disallow: /swift-codes/*/ssi Disallow: /*/swift-codes/*/ssi # swift-codes pages: robots.txt block REMOVED 2026-03-25 # All swift-codes templates now have # Removing the crawl block lets Google SEE the noindex tag and actively reclassify # 157K entries from "crawled not indexed" (hurts quality) to "excluded by noindex" (neutral) # robots.txt block was hiding noindex from Google, causing 60+ week decay vs 4-8 weeks # SSI subpages remain blocked (no noindex tag, thin content): # Bytespider (ByteDance) — not crawling, scraping /sanctions via POST requests User-agent: Bytespider Disallow: /sanctions Sitemap: https://ohmyfin.ai/sitemap.xml