l234244
Sep 4th 2004, 1:52 pm
Can some one give me the low down on why some people put image in their robots.txt file, does it help the spidering? And is it only used when alot of images are within a website?
stephfoster
Sep 5th 2004, 10:02 am
Are they perhaps blocking the images from being spidered? That's very common. The idea is to keep it out of image searches so the images are just a little less likely to be found by some random person and stolen for use on another site. I can't think of any advantage to telling spiders to specifically look at images.
vBulletin® v3.6.8, Copyright ©2000-2008, Jelsoft Enterprises Ltd.