robots/htaccess

Discussion in 'robots.txt' started by Solicitors Mortgages, Apr 21, 2005.

0
  1. #1
    i have some settings on my .htaccess to ban certain areas,
    if i then have a blank"" robots.txt file are those areas still blocked? :confused:
    or are these totally different things ??


    "" = fully spiderable...like this below
    V
    V
    V
    V

    User-agent: *
    Disallow:
     
    Solicitors Mortgages, Apr 21, 2005 IP
  2. ServerUnion

    ServerUnion Peon

    Messages:
    3,611
    Likes Received:
    296
    Best Answers:
    0
    Trophy Points:
    0
    #2
    If you have secured directories, the SE's will not be abe to enter,
     
    ServerUnion, Apr 21, 2005 IP
  3. sarahk

    sarahk iTamer Staff

    Messages:
    29,016
    Likes Received:
    4,584
    Best Answers:
    124
    Trophy Points:
    665
    #3
    search engines "read and respect" robots.txt (or not)
    htaccess is the law

    there's a big difference
     
    sarahk, Apr 21, 2005 IP
  4. gatordun

    gatordun Guest

    Messages:
    114
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Use both robots.txt and htaccess!
     
    gatordun, Jul 29, 2005 IP