Jun 19
22
Gary Illyes from Google added to the big crawl budget article he published on the webmaster blog in January 2017 a new question and answer. It basically says that if you disallow a URL in your robots.txt file, then those URLs do not affect your crawl budget
Visit link:
Google: Disallowed URLs Through Robots.txt Does Not Affect Crawl Budget