1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Will Google Crawl Links On A Robot Disallowed Page?

Discussion in 'robots.txt' started by cartoonbabyboy, Feb 7, 2013.

0
  1. #1
    So I have a page that I don't want to appear in Google Search Results, however on that page are links that I DO want Google to follow. So my question is will Google find those links.
     
    cartoonbabyboy, Feb 7, 2013 IP
  2. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #2
    So instead of using robots.txt to disallow it, you should use the robots meta tag:

    <meta name="robots" content="noindex, follow"/>
    HTML:
    This will do what you want.
     
    ryan_uk, Feb 9, 2013 IP
  3. Reena Yadav

    Reena Yadav Member

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    31
    #3
    You should use HTML <META> tag to tell robots not to index the page, and Google scan it for links to follow.
    For example:
    <head>
    <META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">
    </head>
     
    Reena Yadav, Feb 9, 2013 IP
  4. cartoonbabyboy

    cartoonbabyboy Active Member

    Messages:
    317
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #4
    Oh I see, thank you so much!
     
    cartoonbabyboy, Feb 9, 2013 IP