Jul 14
16
The robots.txt testing tool in Google Webmaster Tools has just received an update to highlight errors causing Google not to crawl pages on your website, let you edit your file, test if URLs are blocked, and also view older robots.txt file versions.
Read more here:
New robots.txt Tester Launches in Google Webmaster Tools