1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Using robots.txt for better search engine placement

Discussion in 'robots.txt' started by jducey1234, Jun 1, 2011.

0
  1. #1
    This happened to me totally by accident, but I was wondering if anyone else has done this. I activated a robot.txt file without knowing it. It was only on for 24 hours, but it was enough to make my site disappear off google. However when I removed it, my site was 15 spots higher in ranking, and it has stayed there for the last week.

    I have been working hard to build links etc. but my site hadn't budged. Has anyone else done this to "force" google to shuffle the order?
     
    jducey1234, Jun 1, 2011 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    I would not say that this is a standard practice to shuffle the positions and I would not recommend anyone to even try this out. The reason your website got bump in the rankings has nothing to do with robots.txt, it is mere coincidence. The only reason your website got better positions is because you had worked hard to build the profiles and results started coming out at the time you removed robots.txt.
     
    manish.chauhan, Jun 3, 2011 IP
  3. jelaine

    jelaine Peon

    Messages:
    51
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yes, Google doesn't randomly change their algorithm. It must really because of your link building. Sometimes links are being backlinked later than when you made them. it even took few weeks or even a month to have links backlinked. So you hard work must paid off.
     
    jelaine, Jun 5, 2011 IP
  4. maverick.jones

    maverick.jones Greenhorn

    Messages:
    62
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #4
    Hey i totally agree with you both
     
    maverick.jones, Jun 10, 2011 IP
  5. Manginho

    Manginho Peon

    Messages:
    37
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    yeah that would be a stupid bug...
     
    Manginho, Jun 19, 2011 IP
  6. endu202

    endu202 Peon

    Messages:
    14
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    But the question,how to make good Robot.txt?? Especially for Boost Seo on Site
     
    endu202, Jul 7, 2011 IP
  7. asmartgoat

    asmartgoat Peon

    Messages:
    35
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Is there really such a science. I just put Index Follow and thats it. Now if you look up luke martinez, i'm the first result. Even when there is a famous basketballer with the same name, I have reclaimed my name :)
     
    asmartgoat, Jul 11, 2011 IP
  8. dir18

    dir18 Active Member

    Messages:
    256
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    65
    #8

    robots.txt is a convention to prevent cooperating web crawlers and other web robots from accessing all or part of a website which is otherwise publicly viewable.


    You can't use this for boosting your SEO but you can block google to index unwanted directories,etc.. :)
     
    dir18, Jul 12, 2011 IP
  9. microarm

    microarm Peon

    Messages:
    18
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I don't think robots.txt involve in search engine ranking.
     
    microarm, Jul 15, 2011 IP
  10. gpcola

    gpcola Peon

    Messages:
    11
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    oh it absolutely is if you want to ensure you don't hit duplicate content penalties and such!
     
    gpcola, Jul 24, 2011 IP
  11. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #11
    Just to make it clear to you that robots.txt has nothing to do directly with the rankings of your websites. As dir18 said, it is just set of instructions that let crawlers know what sections of a website they should follow and what to avoid. To improve, you should rather focus on the backlinks.
     
    manish.chauhan, Aug 1, 2011 IP