What its means in Robot.txt file Please help me

Discussion in 'robots.txt' started by skylinkseo, Jan 11, 2013.

0
  1. #1
    Hi DP members,

    Please check following robots and discuss more about specific why they use this way for disallow,

    1. http://www.yellowpages.ca/robots.txt

    Disallow: /search/*-fto*
    Disallow: /search/ad*
    Disallow: /search/*/rcu-*

    2. http://www.quikr.com/robots.txt

    Disallow: *xxxxxxxx*
    Disallow: *Kamapichachi*
    Disallow: *Kamapisachi*
    Disallow: *Kamapisashi*
    Disallow: *xnxx*
    Disallow: *xxnx*
    Disallow: *xnxxx*
    Disallow: *tupe8*
     
    skylinkseo, Jan 11, 2013 IP
  2. DaffodilSW

    DaffodilSW Greenhorn

    Messages:
    57
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    8
    #2
    When you use *, it means everything. For example, this instruction - Disallow: /search/*-fto*. It gives instruction to search engines not to track any url containing '-fto' word, no matter what it has before this word or after.
     
    DaffodilSW, Jan 17, 2013 IP