Robot Txt - Is this the correct way

Discussion in 'Search Engine Optimization' started by Dee2007, Nov 28, 2012.

0
  1. #1
    Hi

    I want to disallow the tell a friend pages on my site, they are formatted like this:

    http://www.sitename.co.uk//Listing/Friend/pagename.html

    But there are other pages that also have the /Listing part that I don't want to block.

    If I add:

    Disallow: /Listing/Friend/

    It won't block the listing folder completely will it? I mean if there is something like /Listing/Mail/ if will still index that.

    I'm sure won't right but just want to double check.

    Thanks

    Dee
     
    Dee2007, Nov 28, 2012 IP
  2. nptifitness

    nptifitness Peon

    Messages:
    239
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    you can set manually on that page using.

    Meta robots tag.


    <html>
    <head>
    <title>...</title>
    <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
    </head>
     
    nptifitness, Nov 28, 2012 IP
  3. Dee2007

    Dee2007 Active Member

    Messages:
    1,185
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    88
    #3
    Hi

    I site is dynamic, so I want to do it via the robot.txt file, the site isn't static html pages, and it's much easier for me to add it to the robot.txt file than start messing around with the php code.

    So if someone can tell me if my method is correct? :)
     
    Dee2007, Nov 28, 2012 IP