A default robots.txt for all my site.......... Please review

Discussion in 'robots.txt' started by danieloffice, Jul 13, 2007.

0
  1. #1
    For my site, say www.pfunny.com, the following is what I put in robots.txt
    The Function is to handle the 404 issue, and also have a auto sitemap.xml for Googlebot
    Is that OK? please comment. Thanks




    Sitemap: http://www.pfunny.com/sitemap.xml
    User-agent: *
    Disallow:
     
    danieloffice, Jul 13, 2007 IP
  2. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #2
    why dont you submit your sitemap from your webmaster tools panel?

    i think it's easy way
     
    trichnosis, Aug 7, 2007 IP