View Full Version : What is a Robot???
eathosting.com
Dec 2nd 2007, 12:52 am
I am trying to figure out what exactly a robot is and how it would benefit my site???
ajsa52
Dec 2nd 2007, 1:03 am
Web site owners use the /robots.txt file to give instructions about their site to web robots.
Web Robots (also known as Web Wanderers, Crawlers, or Spiders), are programs that traverse the Web automatically. Search engines such as Google use them to index the web content, spammer use them to scan for email addresses, and they have many other uses.
Look at these page for additional info:
http://www.robotstxt.org/
http://en.wikipedia.org/wiki/Robots.txt
jmorti
Dec 3rd 2007, 11:33 pm
Do some searches for explanations. That always seems to be the best and fastest way to learn new things on the internet for me.
anantk
Dec 4th 2007, 9:52 am
hey, what's google for ?
SSANZ
Dec 9th 2007, 4:23 am
hey, what's google for ?
Would that be a serious question?
on the robotstxt.org site you will find all the answers
eathosting.com
Dec 12th 2007, 7:44 am
ok thanks guys that helps alot.
gab1982
Dec 20th 2007, 9:58 pm
I was going to ask the same thing, isn't there some benefits and disadvantages in regards to SEO and robots?
vBulletin® v3.6.8, Copyright ©2000-2008, Jelsoft Enterprises Ltd.