Will Google Crawl Links On A Robot Disallowed Page?

Discussion in 'robots.txt' started by cartoonbabyboy, Feb 7, 2013.

0
  1. #1
    So I have a page that I don't want to appear in Google Search Results, however on that page are links that I DO want Google to follow. So my question is will Google find those links.
     
    cartoonbabyboy, Feb 7, 2013 IP
  2. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #2
    So instead of using robots.txt to disallow it, you should use the robots meta tag:

    <meta name="robots" content="noindex, follow"/>
    HTML:
    This will do what you want.
     
    ryan_uk, Feb 9, 2013 IP
  3. Reena Yadav

    Reena Yadav Member

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    31
    #3
    You should use HTML <META> tag to tell robots not to index the page, and Google scan it for links to follow.
    For example:
    <head>
    <META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">
    </head>
     
    Reena Yadav, Feb 9, 2013 IP
  4. cartoonbabyboy

    cartoonbabyboy Active Member

    Messages:
    317
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #4
    Oh I see, thank you so much!
     
    cartoonbabyboy, Feb 9, 2013 IP