Hi friends, Is there any specific purpose to use robots.txt file? I think we can use this to hide some unnecessary elements to search engine and to minimise the page load time. Waiting for different responses. Thanks & Regards, Jones
you can use css and javascript file externally and import link can use which result reduce size of page
dont use flash, also you can add an add on for speed your loading time. You can find this add on at Google webmaster tool
You can use zlib compression for minimizing the pageload robot.txt is just to prevent/allow search bots
If you are running a script, look for some sort of caching plugin. I use w3-totalcache for wordpress w/ memcache and it loads very quickly.