Help Required

[warley]

If a site had the following robots.txt file what would this mean exactly i have been told this site is disallowing links is this true?

User-agent: ia_archiver
Disallow: /

User-agent: Googlebot
Disallow: /frli.htm
Disallow: /frre.htm
Disallow: /passw
Disallow: /ref
Disallow: /lin
Disallow: /free
Disallow: /cgi
Disallow: /impressum.htm
Disallow: /topframebwinticker.htm
Disallow: /send
Disallow: /mail
Disallow: /ext
Disallow: /lin
Disallow: /lm
Disallow: /sgi
Disallow: /bg

Any help appreciated

[nessie]

Looks like they don't like Google :)

They are blocking Google spider from accessing those files & directories listed after "User-agent: Googlebot"

ia_archiver is alexa's bot. It's blocked for the whole site.

[warley]

Oh dear fair enough thanks a lot for that.