## DEVELOPMENT RELATED SETTINGS ## Do not crawl development files and folders: CVS, svn directories and dump files User-agent: * ## Crawl-delay: 10 Disallow: /CVS Disallow: /*.svn$ Disallow: /*.idea$ Disallow: /*.sql$ Disallow: /*.tgz$ ## GENERAL SETTINGS ## SEO IMPROVEMENTS ## Do not crawl sub category pages that are sorted or filtered. Disallow: /resources/blog/?sf_paged* ##Blocking valueless URLs Disallow: /disclosure/ Disallow: /newsletter-disclaimer/ Disallow: /privacy-policy/ Disallow: /form-crs/ Disallow: /wp-admin/ Disallow: /wp-content/plugins/* ## Do not crawl 2-nd home page copy (example.com/index.php/). Uncomment it only if you activated SEO URLs. ## Disallow: /index.php/ ## SERVER SETTINGS Disallow: /wp-admin/ ## IMAGE CRAWLERS SETTINGS ## Extra: Uncomment if you do not wish Google and Bing to index your images # User-agent: Googlebot-Image # Disallow: / # User-agent: msnbot-media # Disallow: / User-agent: GoogleImageProxy Allow: / User-agent: Googlebot-Image Allow: / User-agent: SiteAuditBot Allow: / Sitemap: https://realinvestmentadvice.com/sitemap.xml