1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

NoFollow links from robots.txt

Discussion in 'robots.txt' started by Fracisc, Aug 15, 2010.

  1. #1
    I have it like this:

    User-agent: *
    Allow: /
    Disallow: /article.html
    Code (markup):
    My question is: Will Google follow the links that are in that page? I want google to index that page but do not follow the links.
     
    Fracisc, Aug 15, 2010 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    No this would not work for you. You should rather try following meta tag in the page code:

    <META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW" />
     
    manish.chauhan, Aug 16, 2010 IP
  3. Fracisc

    Fracisc Well-Known Member

    Messages:
    3,670
    Likes Received:
    10
    Best Answers:
    1
    Trophy Points:
    195
    #3
    I do not want to use meta tags, it is easier with robots.txt.

    What does that command do then?
     
    Fracisc, Aug 16, 2010 IP
  4. MirVB

    MirVB Peon

    Messages:
    70
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    It does what you want it to do, index the page, but not follow the links.

    Right now, your robots.txt does not allow articles.html to be crawled or indexed. You should use the meta tag, or use the nofollow attribute on each individual link. The meta tag will be easiest.
     
    MirVB, Aug 16, 2010 IP
  5. Fracisc

    Fracisc Well-Known Member

    Messages:
    3,670
    Likes Received:
    10
    Best Answers:
    1
    Trophy Points:
    195
    #5
    So "disallow" means no index and no follow. Right?
     
    Fracisc, Aug 16, 2010 IP
  6. zigojacko

    zigojacko Member

    Messages:
    33
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    41
    #6
    The robots.txt is to stop (disallow) search engines visiting certain pages of your website, if they can't visit the page, then they can't index it.

    As others have said, for allowing pages to be indexed but not following any links on the page, the robots meta tag should be used.
     
    zigojacko, Aug 17, 2010 IP
  7. seo.lover

    seo.lover Peon

    Messages:
    132
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    thank you very much
     
    seo.lover, Aug 29, 2010 IP
  8. ravishanker

    ravishanker Peon

    Messages:
    17
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Info About Robot.txt and nofollow..

    Robot.txt – This is a file written and stored in the root directory of a website that restricts search engine spiders from indexing certain areas of the website.

    nofollow: It is an HTML attribute value used to instruct some search engines that a hyperlink should not influence the link target's ranking in the search engine's index. It is intended to reduce the effectiveness of certain types of search engine spam, thereby improving the quality of search engine results and preventing spamdexing from occurring.
     
    ravishanker, Sep 1, 2010 IP