Meta Robots Tag - How to Use and When

This is an interesting tag in SEO, which is used by a very few.

The tag comes in the head part of an HTML document just like other usual meta description and keyword tags.

The typical use is
META NAME='ROBOTS' CONTENT='INDEX, FOLLOW, NOARCHIVE'

This says the robots, to index the page (i.e get the page informations and make it available for search results), follow the links (i.e. it can index the urls found as links in the page), and..

NOARCHIVE - it says the crawler not to show the 'cached link' after site details in the search results.

What does it mean to you??

This noarchive may be used by webmasters to hide the information presented to google. This is closely associated with presenting a different page content to crawlers and showing different page content to users. But always it cannot be so.
Some webmasters make the pages which requires user login, crawlable to google. Now user has two option to access the page - either by logging in or seeing the google cache of the page. This provides some propblem for the webmasters and in order to overcome that they may use noarchive.

Here the page is indexed just like a normal page but it will not be shown to public in the cache.

If anybody wants to use this tag specifically to google and leave other crawlers, they can use

META NAME='GOOGLEBOT' CONTENT='INDEX, FOLLOW, NOARCHIVE'

The other two parameters - INDEX and FOLLOW can also be put to their negative ones - NOINDEX and NOFOLLOW

NOINDEX says the crawler not to index the page at all
NOFOLLOW says the craler not to follow the links on the page.



META NAME='ROBOTS' CONTENT='INDEX, NOFOLLOW'

This says the robot to index the page and not to follow the links in it.

So this tag can be used for any page so as to serve a definite purpose.

Hope this information was useful to you.

Thanks

SEOLion