1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Hidden pages

Discussion in 'Search Engine Optimization' started by ~Bored~, Feb 19, 2012.

  1. #1
    I have created a page for the moderators at my site. An html page. Now that page is hidden meaning there are no links pointing to it, if that page gets indexed even though i have the <meta name="robots" content="noindex, nofollow" /> tag, if someone posts the link to that page on a forum or social network site, will i be penalized for "hidden pages"? Or is this ok?
     
    ~Bored~, Feb 19, 2012 IP
  2. SearchBliss

    SearchBliss Well-Known Member

    Messages:
    1,899
    Likes Received:
    70
    Best Answers:
    2
    Trophy Points:
    195
    Digital Goods:
    1
    #2
    It's fine, don't worry about it.
     
    SearchBliss, Feb 19, 2012 IP
  3. RadioBounce

    RadioBounce Banned

    Messages:
    4,171
    Likes Received:
    16
    Best Answers:
    1
    Trophy Points:
    0
    #3
    You should restrict access to it by using robots.txt.
     
    RadioBounce, Feb 19, 2012 IP
  4. ~Bored~

    ~Bored~ Active Member

    Messages:
    109
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #4
    Thanks for the input, so this is not what it means when SEO says a big no no on hidden pages?

    MVG, im afraid to put in the wrong code in the robots.txt and end up restricting other pages. Any guides on that?
     
    ~Bored~, Feb 19, 2012 IP
  5. ~Bored~

    ~Bored~ Active Member

    Messages:
    109
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #5
    Sorry for double posting but will this work? Is this the right way to all bots can crawl all pages except /vipchat.html ?

    User-agent: *
    Disallow: /vipchat.html
    Allow: /
     
    ~Bored~, Feb 19, 2012 IP
  6. RadioBounce

    RadioBounce Banned

    Messages:
    4,171
    Likes Received:
    16
    Best Answers:
    1
    Trophy Points:
    0
    #6
    
    User-agent: *
    Disallow: /vipchat.html
    Code (markup):
     
    RadioBounce, Feb 19, 2012 IP
  7. ~Bored~

    ~Bored~ Active Member

    Messages:
    109
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #7
    Thanks the google toolbar generated this one for me:

    
    User-agent: *
    Disallow: /vipchat.html
    Allow: /
    
    User-agent: Mediapartners-Google
    Allow: /
    
    Code (markup):
    It looks right?
     
    ~Bored~, Feb 19, 2012 IP
  8. RadioBounce

    RadioBounce Banned

    Messages:
    4,171
    Likes Received:
    16
    Best Answers:
    1
    Trophy Points:
    0
    #8
    Looks fine.
     
    RadioBounce, Feb 19, 2012 IP
  9. forumhookers

    forumhookers Peon

    Messages:
    77
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Right mate!
     
    forumhookers, Feb 19, 2012 IP
  10. ~Bored~

    ~Bored~ Active Member

    Messages:
    109
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #10
    Thanks guys for all the support.
     
    ~Bored~, Feb 19, 2012 IP
  11. Canonical

    Canonical Well-Known Member

    Messages:
    2,223
    Likes Received:
    141
    Best Answers:
    0
    Trophy Points:
    110
    #11
    You would be foolish to block the page with Robots.txt.

    1) Anyone looking to hack your forums is going to first look at your robots.txt to see what all you might be 'hiding'. Disallowing the page in robots.txt is just advertising, "Hack this 'secret' page!"
    2) Contrary to popular belief, a disallow in a robots.txt does NOT prevent a page from being shown in Google's search results. If that page has links from external sites, even though they cannot crawl it Google can still show it in the SERPs if they think it is relevant to the search query based on the link text used to link to it. You will know when this has happened when you see a SERPs listing with a title and a URL but it has no snippet. The reason the snippet of text is missing is that Google could not crawl the page. Therefore, they could not see the meta description to determine if it should be shown as the snippet. Nor could they construct a snippet from the content of the page because they could not crawl it.

    You had the right solution to begin with. Simply include a <meta name="robots" content="noindex"> element in the head of the page you do NOT want to show in the SERPs and allow the crawlers to crawl the page (i.e. do NOT disallow the URL in robots.txt). The engines have to be able to crawl the page to find the meta robots noindex to figure out that you do not want them to index or show the page in their SERPs. Once they find the meta robots element they will abide by it. And chances are that no one will ever link to it anyway.
     
    Canonical, Feb 19, 2012 IP
  12. ~Bored~

    ~Bored~ Active Member

    Messages:
    109
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #12
    Thanks for the feedback!
     
    ~Bored~, Feb 19, 2012 IP