Credit Cards - Credit Cards - Free Advertising - Web Advertising - Loans

PDA

View Full Version : Help required for ROBOTS.txt?


nshadab
Jan 22nd 2008, 3:19 am
I am very new to this word...ROBOTS.txt

How this file is helpful to our website as far as the search engines are concern?

Is this helpfull to gain good ranking at search engine? If yes; What important changes we must make, so that we have good ranking?

Any other suggestions; we must take care with regards to ROBORTS.txt.

Thanks

Dr. XYZ
Jan 22nd 2008, 3:39 am
Robots.txt has nothing to do with pagerank. This is a file that helps you to allow/avoid search engines crawl certain directories of files that you can specify.

Learn more about robots.txt here:www.robotstxt.org


ยท

nshadab
Jan 22nd 2008, 4:25 am
Thanks for the update.
What if; i dont want search engines crawl a particular directory.....
and a condition i want search engines crawl a particular directory

is there any particluar keywords to make the search engines crawls a particular directory?

Rivers
Jan 24th 2008, 1:56 am
bots by default are set to crawl everything in your website unless you specified them not to.

to block a directory:

User-agent: *
Disallow: /directoryname/

hope that helps :D

catanich
Feb 2nd 2008, 2:19 pm
Also, you can go to www.robots.org for the full command set.