The CaymanHost Web Hosting Directory

This brief article outlines the importance of using robots.txt files and good sitemaps to help search engines index your website or blog properly.

Having built your website you are probably waiting for that flood of visitors to start arriving but this won’t happen overnight. There is a great deal you can do to improve your chances of good search engine rankings but the whole field of SEO (Search Engine Optimization) can be very complicated and is rarely an exact science. The information is out there if you are prepared to do your research but for many people, time is the biggest limiting factor when it comes to SEO and there is a lot of information to wade through.

There are however a couple of very simple steps that anyone can take to help the Search engines find and index your site correctly. Both will take only a couple of minutes to implement and will repay you well if you take the trouble to complete them.

Firstly, you should ensure that you have a robots.txt file in the root of your domain (the place where your site’s homepage resides) and that it is valid and functioning correctly. The simplest way to do this is to go to one of the many online robots.txt generator and validator sites, where all you will have to do is type in your website address and a couple of options and they will generate the appropriate file for you. We recommend Yellowpipe Robots.txt Generator which will also enable you to choose to banish the robots you don’t want visiting your site, such as known email harvesting bots. There are plenty of other online services available if you search for them. This method ensures an error free robots.txt file which you can then upload to your server. It’s that simple and the search engine spiders will then be aware of exactly where to visit on your site.

The next task on your list is to set up sitemaps, which have become steadily more important in recent times. You may well already have a Google sitemap on your site and if you do, it will probably be called sitemap.xml. Sitemaps also reside in the root of your domain and make it much easier for engines to find and index your content.

There are plenty of ways to set up a Google sitemap and it’s probably the most important one, but it won’t hurt you to have more sitemaps either. The easiest way for you to do this is to use the excellent services at AutoMapIt Automatic Sitemap Service who will not only create six sitemaps automatically for you, but upload them to your server too if you wish. If you prefer to upload the files yourself, you can do so. This is a free service, although you can upgrade at very reasonable cost and benefit from some even more valuable services. Setting up the free sitemap generation is simple and quick and will give you a far greater chance of being fully and properly indexed. Certainly worth a few minutes of your time.

As we said at the outset, these are very basic SEO steps, but nonetheless a good starting point for any webmaster.

Technorati Tags: Free sitemap generator , Sitemaps , Google sitemap , Robots.txt validator , Robots.txt generator , SEO , Building sitemaps , How to build sitemaps , Making a sitemap , Webmaster tools

No Rating

Rate this:

2.5

Filed under: Blogging, SEO, Web Design, Website Promotion