<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>MG Marketing &#187; seo: crawling and robots</title>
	<atom:link href="http://mg-mk.com/en/tag/seo-crawling-and-robots/feed/" rel="self" type="application/rss+xml" />
	<link>http://mg-mk.com/en</link>
	<description>Online Marketing Agency</description>
	<lastBuildDate>Mon, 20 Apr 2026 14:00:00 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.1</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>The ultimate guide to bot herding and spider wrangling — Part Two</title>
		<link>http://mg-mk.com/en/the-ultimate-guide-to-bot-herding-and-spider-wrangling-%e2%80%94-part-two/</link>
		<comments>http://mg-mk.com/en/the-ultimate-guide-to-bot-herding-and-spider-wrangling-%e2%80%94-part-two/#comments</comments>
		<pubDate>Wed, 02 May 2018 15:17:00 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[how to: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/the-ultimate-guide-to-bot-herding-and-spider-wrangling-%e2%80%94-part-two/</guid>
		<description><![CDATA[Next up in a series on bots and why crawl budgets are important, Columnist Stephan Spencer explains how to direct the engine bots to what's important on your site and how to avoid common coding issues. ]]></description>
			<content:encoded><![CDATA[<p>Next up in a series on bots and why crawl budgets are important, Columnist Stephan Spencer explains how to direct the engine bots to what&#8217;s important on your site and how to avoid common coding issues. </p>
<p>View original post here:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/Ut8uE-71Wy8/the-ultimate-guide-to-bot-herding-and-spider-wrangling-part-two-296290" title="The ultimate guide to bot herding and spider wrangling — Part Two">The ultimate guide to bot herding and spider wrangling — Part Two</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/the-ultimate-guide-to-bot-herding-and-spider-wrangling-%e2%80%94-part-two/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>The first steps of your SEO audit: Indexing issues</title>
		<link>http://mg-mk.com/en/the-first-steps-of-your-seo-audit-indexing-issues/</link>
		<comments>http://mg-mk.com/en/the-first-steps-of-your-seo-audit-indexing-issues/#comments</comments>
		<pubDate>Fri, 27 Apr 2018 13:30:00 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/the-first-steps-of-your-seo-audit-indexing-issues/</guid>
		<description><![CDATA[Even a magic SEO wand will not get a web page to rank if the page has not been indexed. Contributor Janet Driscoll Miller suggests that making sure web pages can be indexed is key during an SEO audit. The post The first steps of your SEO audit: Indexing issues appeared first on Search Engine Land. ]]></description>
			<content:encoded><![CDATA[<p>Even a magic SEO wand will not get a web page to rank if the page has not been indexed. Contributor Janet Driscoll Miller suggests that making sure web pages can be indexed is key during an SEO audit. The post The first steps of your SEO audit: Indexing issues appeared first on Search Engine Land. </p>
<p>The rest is here:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/aZj1olmF0No/the-first-steps-of-your-seo-audit-indexing-issues-296853" title="The first steps of your SEO audit: Indexing issues">The first steps of your SEO audit: Indexing issues</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/the-first-steps-of-your-seo-audit-indexing-issues/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Google changes request recrawl limits, noting daily limits versus monthly quotas</title>
		<link>http://mg-mk.com/en/google-changes-request-recrawl-limits-noting-daily-limits-versus-monthly-quotas/</link>
		<comments>http://mg-mk.com/en/google-changes-request-recrawl-limits-noting-daily-limits-versus-monthly-quotas/#comments</comments>
		<pubDate>Mon, 19 Feb 2018 14:13:20 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[google: search console]]></category>
		<category><![CDATA[google: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/google-changes-request-recrawl-limits-noting-daily-limits-versus-monthly-quotas/</guid>
		<description><![CDATA[Google has finally disclosed the new recrawl limits by revising the help document over the weekend. ]]></description>
			<content:encoded><![CDATA[<p>Google has finally disclosed the new recrawl limits by revising the help document over the weekend. </p>
<p>Read the rest here:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/THrZUkO2Ofs/google-changes-request-recrawl-limits-noting-daily-limits-versus-monthly-quotas-292371" title="Google changes request recrawl limits, noting daily limits versus monthly quotas">Google changes request recrawl limits, noting daily limits versus monthly quotas</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/google-changes-request-recrawl-limits-noting-daily-limits-versus-monthly-quotas/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>How to keep your staging or development site out of the index</title>
		<link>http://mg-mk.com/en/how-to-keep-your-staging-or-development-site-out-of-the-index/</link>
		<comments>http://mg-mk.com/en/how-to-keep-your-staging-or-development-site-out-of-the-index/#comments</comments>
		<pubDate>Wed, 22 Nov 2017 16:45:46 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>
		<category><![CDATA[seo: domains & urls]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/how-to-keep-your-staging-or-development-site-out-of-the-index/</guid>
		<description><![CDATA[You generally wouldn't want your staging site appearing in search results, so how can you prevent Google from indexing this content? Columnist Patrick Stox offers some tips. The post How to keep your staging or development site out of the index appeared first on Search Engine Land. ]]></description>
			<content:encoded><![CDATA[<p>You generally wouldn&#8217;t want your staging site appearing in search results, so how can you prevent Google from indexing this content? Columnist Patrick Stox offers some tips. The post How to keep your staging or development site out of the index appeared first on Search Engine Land. </p>
<p>Originally posted here:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/NQulhRJ5K4E/how-to-keep-staging-development-site-index-286987" title="How to keep your staging or development site out of the index">How to keep your staging or development site out of the index</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/how-to-keep-your-staging-or-development-site-out-of-the-index/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>SEO + UX = Success</title>
		<link>http://mg-mk.com/en/seo-ux-success/</link>
		<comments>http://mg-mk.com/en/seo-ux-success/#comments</comments>
		<pubDate>Fri, 17 Nov 2017 21:48:53 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>
		<category><![CDATA[seo: engagement]]></category>
		<category><![CDATA[seo: mobile search]]></category>
		<category><![CDATA[seo: site speed]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/seo-ux-success/</guid>
		<description><![CDATA[These days, SEO best practices include consideration for the user experience (UX). ]]></description>
			<content:encoded><![CDATA[<p>These days, SEO best practices include consideration for the user experience (UX). </p>
<p>Link:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/d7o1W24uL-Y/seo-ux-success-286638" title="SEO + UX = Success">SEO + UX = Success</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/seo-ux-success/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>The final recrawl analysis: A powerful and important last step after implementing large-scale SEO changes</title>
		<link>http://mg-mk.com/en/the-final-recrawl-analysis-a-powerful-and-important-last-step-after-implementing-large-scale-seo-changes/</link>
		<comments>http://mg-mk.com/en/the-final-recrawl-analysis-a-powerful-and-important-last-step-after-implementing-large-scale-seo-changes/#comments</comments>
		<pubDate>Thu, 16 Nov 2017 17:02:58 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>
		<category><![CDATA[seo: general]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/the-final-recrawl-analysis-a-powerful-and-important-last-step-after-implementing-large-scale-seo-changes/</guid>
		<description><![CDATA[You've crawled your client's site, identified the SEO issues and fixed those issues in staging. Great job! But columnist Glenn Gabe shares why recrawling the site is critical once your SEO changes have been pushed live. The post The final recrawl analysis: A powerful and important last step after... ]]></description>
			<content:encoded><![CDATA[<p>You&#8217;ve crawled your client&#8217;s site, identified the SEO issues and fixed those issues in staging. Great job! But columnist Glenn Gabe shares why recrawling the site is critical once your SEO changes have been pushed live. The post The final recrawl analysis: A powerful and important last step after&#8230; </p>
<p>Original post:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/u3gIXSFIz40/final-recrawl-analysis-powerful-important-last-step-implementing-large-scale-seo-changes-286675" title="The final recrawl analysis: A powerful and important last step after implementing large-scale SEO changes">The final recrawl analysis: A powerful and important last step after implementing large-scale SEO changes</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/the-final-recrawl-analysis-a-powerful-and-important-last-step-after-implementing-large-scale-seo-changes/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>How we hacked the Baidu link submission script for better indexation</title>
		<link>http://mg-mk.com/en/how-we-hacked-the-baidu-link-submission-script-for-better-indexation/</link>
		<comments>http://mg-mk.com/en/how-we-hacked-the-baidu-link-submission-script-for-better-indexation/#comments</comments>
		<pubDate>Fri, 03 Nov 2017 17:21:28 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[baidu]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[search marketing]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>
		<category><![CDATA[seo: domains & urls]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/how-we-hacked-the-baidu-link-submission-script-for-better-indexation/</guid>
		<description><![CDATA[Baidu's Link Submission Script is a great way to help Baidu index your content, but it can create some potential SEO issues. ]]></description>
			<content:encoded><![CDATA[<p>Baidu&#8217;s Link Submission Script is a great way to help Baidu index your content, but it can create some potential SEO issues. </p>
<p>Read the original:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/ZzGb_9a5TzI/hacked-baidu-link-submission-script-better-indexation-284819" title="How we hacked the Baidu link submission script for better indexation">How we hacked the Baidu link submission script for better indexation</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/how-we-hacked-the-baidu-link-submission-script-for-better-indexation/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>The four pillars of an effective SEO strategy</title>
		<link>http://mg-mk.com/en/the-four-pillars-of-an-effective-seo-strategy/</link>
		<comments>http://mg-mk.com/en/the-four-pillars-of-an-effective-seo-strategy/#comments</comments>
		<pubDate>Wed, 01 Nov 2017 20:00:40 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[link building]]></category>
		<category><![CDATA[link building: general]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>
		<category><![CDATA[seo: general]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/the-four-pillars-of-an-effective-seo-strategy/</guid>
		<description><![CDATA[Search engine optimization (SEO) can be complex, but columnist Marcus Miller simplifies it by breaking down its key elements into four main categories. ]]></description>
			<content:encoded><![CDATA[<p>Search engine optimization (SEO) can be complex, but columnist Marcus Miller simplifies it by breaking down its key elements into four main categories. </p>
<p>See the original post here:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/Iu3TtRfOuBA/four-pillars-effective-seo-strategy-285584" title="The four pillars of an effective SEO strategy">The four pillars of an effective SEO strategy</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/the-four-pillars-of-an-effective-seo-strategy/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Tips to troubleshoot your technical SEO</title>
		<link>http://mg-mk.com/en/tips-to-troubleshoot-your-technical-seo/</link>
		<comments>http://mg-mk.com/en/tips-to-troubleshoot-your-technical-seo/#comments</comments>
		<pubDate>Thu, 26 Oct 2017 17:54:16 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>
		<category><![CDATA[seo: domains & urls]]></category>
		<category><![CDATA[seo: duplicate content]]></category>
		<category><![CDATA[seo: general]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/tips-to-troubleshoot-your-technical-seo/</guid>
		<description><![CDATA[Stumped by a technical SEO issue? Columnist Patrick Stox has some tips and tricks to help you diagnose and solve some common problems. The post Tips to troubleshoot your technical SEO appeared first on Search Engine Land]]></description>
			<content:encoded><![CDATA[<p>Stumped by a technical SEO issue? Columnist Patrick Stox has some tips and tricks to help you diagnose and solve some common problems. The post Tips to troubleshoot your technical SEO appeared first on Search Engine Land</p>
<p>Read more:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/iVJDJssdD5c/tips-make-better-technical-seo-285329" title="Tips to troubleshoot your technical SEO">Tips to troubleshoot your technical SEO</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/tips-to-troubleshoot-your-technical-seo/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>19 technical SEO facts for beginners</title>
		<link>http://mg-mk.com/en/19-technical-seo-facts-for-beginners/</link>
		<comments>http://mg-mk.com/en/19-technical-seo-facts-for-beginners/#comments</comments>
		<pubDate>Mon, 23 Oct 2017 16:14:28 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[SEO]]></category>
		<category><![CDATA[all things seo column]]></category>
		<category><![CDATA[channel: seo]]></category>
		<category><![CDATA[seo: crawling and robots]]></category>
		<category><![CDATA[seo: domains & urls]]></category>
		<category><![CDATA[seo: image search]]></category>
		<category><![CDATA[seo: mobile search]]></category>
		<category><![CDATA[seo: site speed]]></category>

		<guid isPermaLink="false">http://mg-mk.com/en/19-technical-seo-facts-for-beginners/</guid>
		<description><![CDATA[Want to dip your toes into techncial SEO? Columnist John Lincoln shares some fun tidbits for those just starting out. The post 19 technical SEO facts for beginners appeared first on Search Engine Land]]></description>
			<content:encoded><![CDATA[<p>Want to dip your toes into techncial SEO? Columnist John Lincoln shares some fun tidbits for those just starting out. The post 19 technical SEO facts for beginners appeared first on Search Engine Land</p>
<p>See more here:<br />
<a target="_blank" href="http://feeds.searchengineland.com/~r/searchengineland/~3/3rWD5bhltKc/19-technical-seo-facts-beginners-282949" title="19 technical SEO facts for beginners">19 technical SEO facts for beginners</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mg-mk.com/en/19-technical-seo-facts-for-beginners/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
