Sep 12
27
Twitter recently updated its robots.txt file and, though the change opens up millions of pages to being crawled, there’s no guarantee that the main search engines want what Twitter is offering. The Sociable seems to have been the first to notice Twitter’s robots.txt changes, which now… Please visit Search Engine Land for the full article
Visit link:
Twitter Opens Up To More Crawling, But Do Search Engines Want Its Search Results In Theirs?