1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

The robots.txt file in my site does not work?

Discussion in 'robots.txt' started by peterzhou, Aug 5, 2010.

0
  1. #1
    I wrote this in the file:

    User-agent:*
    Disallow: /cache/
    Disallow: /api/
    Disallow: /index.php?

    And i am sure i put it in the root directory. but google still indexed my pages including /index.php? in the url.

    Is it wrong with the CHMOD of the file? I set it to 777, how to set it?

    Thanks
     
    peterzhou, Aug 5, 2010 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    When did you update your robots.txt? If you disallow crawling of a URL, this does not automatically remove the URL from the search results, but stop crawling the restricted url. Google takes some time to remove the disallowed URL from the search results. If you really wish to remove that URL from the search results, you can remove it through site removal tool in webmasters tool.
     
    manish.chauhan, Aug 6, 2010 IP
  3. Sketch5646

    Sketch5646 Peon

    Messages:
    11
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    im in the same situation !!!
     
    Sketch5646, Sep 28, 2010 IP
  4. Sudhaa

    Sudhaa Peon

    Messages:
    25
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    You can also add meta robots tag with nofollow and no index to the page.
     
    Sudhaa, Sep 28, 2010 IP