Create robots.txt File, Need Your Suggestion

Discussion in 'robots.txt' started by prclicks, Jun 3, 2011.

0
  1. #1
    For Wordpress Blog:
    I have create robots.txt file, I got ideas for which file should be crawl and which file should not.
    QUERY:
    mysite.com/webalizer:
    I am confuse about this file that it should be crawl or not.
    QUERY: YOUR VIEW:
    Que: Which file of the blog should be crawl which have important requirement in Search Engine Ranking?

    FOR PHPBB FORUM:

    Query:
    Which are the file for forum that should not be crawl of search engine?

    Thanks in Advance:
     
    prclicks, Jun 3, 2011 IP