My lovely clients often ask me questions about Search Engine Optimisation (SEO).
I figured you all might be interested in the answers, and so each week I’ll be writing a post to answer a recent question.
Don’t worry. They’ll be short and snappy. And I’ll do my best to answer the question in a non-geeky way that everyone can understand.
Okay, here we go. This one from one of my agency chums.
SEO question:
Do I need to include a link to my XML sitemap in my robots.txt file?
Short answer:
Yes.
Long answer:
Yes. Adding a link to the XML sitemap helps the crawlers/spider bots find it more easily. It’s particularly useful if you’re running multiple sitemaps (one for mobiles, one for images, one for videos, etc.)
And Google tell us
“This file lets Google and other search engines learn the organisation of your site content to more intelligently crawl your site.”
They rarely state anything directly, so when they do you’d better take notice.
Adding a link in robots.txt to your XML sitemap also means you don’t have to create a Webmaster Tools account for every search engine on the planet to submit it.
Sure, it’s not important for Search Engine X if you’ve already submitted it through Search Engine X’s Webmaster Tools. But it can be very important if you rely on traffic from search engines you do not have a Webmaster Tools account for.
One more point (forgive me if it’s too obvious): /sitemap.xml is just a common naming convention. Your robots.txt file can list more XML sitemaps (with different names and paths).
Got an SEO question?
If you’ve got a question about SEO, why not email it to me or post it on my Google+ or Facebook page?
You can even Tweet me if you feel so inclined.
