1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How to NOT get indexed?

Discussion in 'Google' started by alexs464, Aug 24, 2008.

  1. #1
    Hello, folks!

    I have a page that I don't want to get indexed. I added the

    <meta name="robots" content="no index,no follow"> but the page still got indexed... What would be the easiest way to remove it from Google/prevent it from getting indexed?

    Thanks,

    Alex
     
    alexs464, Aug 24, 2008 IP
  2. tattoos

    tattoos Prominent Member

    Messages:
    1,903
    Likes Received:
    150
    Best Answers:
    0
    Trophy Points:
    335
    #2
    Robots.txt

    User-Agent: *
    Disallow: /page-name.html
    Disallow: /page-name2.html
    Allow: /

    That would probably be the best way..
    Add another Disallow: rule on a new line for every page you don't want indexed!
    Or: Disallow: /subdirectory/ to block a whole directory

    Cheers
    James
     
    tattoos, Aug 24, 2008 IP
  3. alexs464

    alexs464 Well-Known Member

    Messages:
    1,357
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    125
    #3
    Sorry but I am a complete idiot when it comes to... I am shamed to admit, even basic html... SO where exactly do I add this?

    Or do I create a Robot.txt document, with content being:
    User-Agent: *
    Disallow: /page-name.html
    Allow: /

    and then update it to the server?

    Thank You!
     
    alexs464, Aug 24, 2008 IP
  4. supergoz

    supergoz Peon

    Messages:
    86
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    There has to be a file named robots.txt at the root of the server. If not you will have to create it with a text editor. Basically start a new file, write the pages or directories you do not want indexed and copy the file to the root of the server. Search engines follow the instructions in the robots.txt
     
    supergoz, Aug 24, 2008 IP
  5. tattoos

    tattoos Prominent Member

    Messages:
    1,903
    Likes Received:
    150
    Best Answers:
    0
    Trophy Points:
    335
    #5
    Yep! you got it..
    here ya go.
    Right click and save as.
    http://www.uvrx.com/alexs464.txt

    Change page-name.html to the page you want blocked, and change the name of the file to robots.txt then upload it to the root directory :D

    Cheers
    James
     
    tattoos, Aug 24, 2008 IP
    alexs464 likes this.
  6. wgn_white

    wgn_white Active Member

    Messages:
    130
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    73
    #6
    Addition...

    Go to Google Webmaster Tools account and remove the url that have been indexed.:rolleyes:
     
    wgn_white, Aug 24, 2008 IP
    alexs464 likes this.
  7. alexs464

    alexs464 Well-Known Member

    Messages:
    1,357
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    125
    #7
    Thanks a lot!

    Reps added :)
     
    alexs464, Aug 24, 2008 IP
  8. AngelaE8654

    AngelaE8654 Active Member

    Messages:
    935
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    85
    #8
    Yeah, me too and I'm watching this thread with interest. I had the same question and I am just as untechie as you are.

     
    AngelaE8654, Aug 24, 2008 IP
  9. tattoos

    tattoos Prominent Member

    Messages:
    1,903
    Likes Received:
    150
    Best Answers:
    0
    Trophy Points:
    335
    #9
    You should only do it this way if you are sure you do not want the page indexed anytime within the next 6 months.. you have to be very careful doing that.. if you accidentally remove a main folder, then you end up wiping your whole site out of the index for a very long time. "not recommended"

    James
     
    tattoos, Aug 24, 2008 IP
  10. alexs464

    alexs464 Well-Known Member

    Messages:
    1,357
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    125
    #10
    alexs464, Aug 24, 2008 IP
  11. wgn_white

    wgn_white Active Member

    Messages:
    130
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    73
    #11
    Ah? :eek:

    I read the first post that he seems to be desperately don't want the page to be indexed...

    Maybe it's a download page or what...

    But the remove url is per url isn't it?
    It's only remove http://domain.com/theurl and not http://domain.com

    Correct me if I'm wrong...
    I never removed any url actually...
    Why would we?

    If the page is a page like download page, we can use password protect, can we?
     
    wgn_white, Aug 24, 2008 IP
  12. alexs464

    alexs464 Well-Known Member

    Messages:
    1,357
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    125
    #12
    I do use password protect for other pages.
    But this particular page - I don't want to password protect it. I just want it not indexed for a few weeks... because later on I will change the content and I WOULD want it to be indexed.
     
    alexs464, Aug 24, 2008 IP
  13. wgn_white

    wgn_white Active Member

    Messages:
    130
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    73
    #13
    Ah, I see...
    Then you SHOULD NOT remove it from Google Webmasters Tools...

    But, if it's already indexed, people still find it anyway, "accidentally"...
     
    wgn_white, Aug 24, 2008 IP
  14. tattoos

    tattoos Prominent Member

    Messages:
    1,903
    Likes Received:
    150
    Best Answers:
    0
    Trophy Points:
    335
    #14
    Thanks!


    if the page is in the root folder then just like in the robots.txt. EG: Disallow: /page-name.html
    If the page is in a sub folder then
    Disallow: /subfolder-name/page-name.html

    if you went..
    Disallow: /subfolder/
    and did not supply a page-name.html, you would block all pages in that subfolder.

    You can also include this on the last line.
    sitemap: http://www.yourwebsite.com/sitemap.xml
    Code (markup):
    (include the full address.)
    That will tell the robots where to find your sitemap :)

    So in the end it will look like this

    User-Agent: *
    Disallow: /page-name.html
    Allow: /
    sitemap: http://www.yourwebsite.com/sitemap.xml
    Code (markup):
    Cheers
    James
     
    tattoos, Aug 24, 2008 IP
  15. lightlysalted

    lightlysalted Active Member

    Messages:
    2,067
    Likes Received:
    32
    Best Answers:
    0
    Trophy Points:
    90
    #15

    Great reply, good old robots.txt files are really helpful for this
     
    lightlysalted, Aug 24, 2008 IP
  16. jamesplato

    jamesplato Active Member

    Messages:
    359
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    60
    #16
    if it's already indexed, you could add a redirect until it is ready for primetime to be indexed:

    in the head tags of the page add:
    
    
    <meta http-equiv="refresh" content="0; url=http://www.domain.com/otherpage.html">
    
    
    Code (markup):
     
    jamesplato, Aug 24, 2008 IP
    monfis likes this.
  17. monfis

    monfis Well-Known Member

    Messages:
    1,476
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    160
    #17
    Nice one! Rep added. :)
     
    monfis, Aug 24, 2008 IP