Yahoo - Supporting wildcards in robots.txt

Discussion in 'Yahoo' started by dhruv37, Nov 7, 2006.

  1. #1
    Yahooo search crawler now supporting wildcards in robots.txt

    We can now use '*' in robots directives for Yahoo! Slurp to wildcard match a sequence of characters in URL. We can use this symbol in any part of the URL string we provide in the robots directive.

    You can read more on - http://www.ysearchblog.com/archives/000372.html
     
    dhruv37, Nov 7, 2006 IP
  2. pthona

    pthona Active Member

    Messages:
    327
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    58
    #2
    that good to hear

    thanks
     
    pthona, Nov 14, 2006 IP