Sitemap: https://www.octoparse.com/sitemap.xml

User-agent: *
Disallow: /console/
Disallow: /subscribe
Disallow: /auth
Disallow: /share
Disallow: /*/feed/
Disallow: /cdn-cgi/