Disallowing feed?

Discussion in 'robots.txt' started by dmi, Jul 24, 2009.

  1. #1
    Is it necessary? Should I disallow the /feed ?
     
    dmi, Jul 24, 2009 IP
  2. rhewitt

    rhewitt Member

    Messages:
    113
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    30
    #2
    I see no reason to disallow it. Some rss crawlers might pick up on it too, possibly bringing you more traffic.
     
    rhewitt, Jul 26, 2009 IP
  3. jabz.biz

    jabz.biz Active Member

    Messages:
    384
    Likes Received:
    6
    Best Answers:
    1
    Trophy Points:
    70
    #3
    I always disallow search engines for my feeds. Search engine crawlers have no business there. I also do not link to my feed directly...I remember a time when this messed up Google. That was long ago, but who knows what crazy search engines are out there. Save the traffic and keep the bots out. You get traffic from pinging, not crawling.
     
    jabz.biz, Jul 28, 2009 IP