SEO benefits allowing spiders to crawl profiles vs. no crawl profiles on messageboard

Discussion in 'Search Engine Optimization' started by verbs, Jul 29, 2007.

  1. #1
    My current setup on my messageboard is that I do not allow guests/unregistered members to view profiles....so this includes search engine spiders. I see all the spiders trying to crawl to my members' profiles, but they get stopped with error messages since guests aren't allowed to access it. I did an rel=no follow to the profiles, but I figured I didn't want the spiders spending a lot of time crawling profiles which I felt was useless information to crawl.

    Am I shooting myself in the foot by doing this? Will the error messages the spiders see have a negative impact?
     
    verbs, Jul 29, 2007 IP
  2. The_FoX

    The_FoX Peon

    Messages:
    666
    Likes Received:
    21
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Nope not at all. You can also block them using the robots.txt file

    Give this command in the robots file

    Disallow: /yourfiledirectory/
     
    The_FoX, Jul 30, 2007 IP
  3. sweetfunny

    sweetfunny Banned

    Messages:
    5,743
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #3
    The good part is you dont waste crawl budget and cut down the number of links per thread to non-content rich pages.

    The bad part is members may want their profile indexed with PR.

    You really only want your valuable content recieving the Google attention. Its also a good idea to block your calendar.php as its also 1,000s of pages of duplicate junk.
     
    sweetfunny, Jul 30, 2007 IP
  4. verbs

    verbs Peon

    Messages:
    245
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4

    Yes, I blocked calendar, and have been blocking profiles but was debating on letting spiders crawl members' profiles but with a no-follow tag on profiles.
     
    verbs, Aug 2, 2007 IP
  5. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #5
    i dont think they will make profit with allowing search engine to crawl the profiles on a forum
     
    trichnosis, Aug 3, 2007 IP