1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

What its means in Robot.txt file Please help me

Discussion in 'robots.txt' started by skylinkseo, Jan 11, 2013.

0
  1. #1
    Hi DP members,

    Please check following robots and discuss more about specific why they use this way for disallow,

    1. http://www.yellowpages.ca/robots.txt

    Disallow: /search/*-fto*
    Disallow: /search/ad*
    Disallow: /search/*/rcu-*

    2. http://www.quikr.com/robots.txt

    Disallow: *xxxxxxxx*
    Disallow: *Kamapichachi*
    Disallow: *Kamapisachi*
    Disallow: *Kamapisashi*
    Disallow: *xnxx*
    Disallow: *xxnx*
    Disallow: *xnxxx*
    Disallow: *tupe8*
     
    skylinkseo, Jan 11, 2013 IP
  2. DaffodilSW

    DaffodilSW Greenhorn

    Messages:
    57
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    8
    #2
    When you use *, it means everything. For example, this instruction - Disallow: /search/*-fto*. It gives instruction to search engines not to track any url containing '-fto' word, no matter what it has before this word or after.
     
    DaffodilSW, Jan 17, 2013 IP