Ok, i have a site, but i'm pretty much an amatuer when it comes to the likes of HTML and CSS, but one thing i would like to know is, is there a code that i can install that will allow all robots to index at here http://rugbyleaguehq.com/ I notice that there are few links indexed on google, with more on yahoo and a couple of others, but none at all on MSN? http://www.submitexpress.com/cgi-bin/linkpoptest/popular.pl?url1=rugbyleaguehq.com&url2=&url3=&email= Is there a code out there that will allow all robots to index, or am i just out of luck? And please, remember that i am an amatuer, and too much of a technical explanation will only confuse me
By default ALL robots are allowed to browse your site. unless you have a robots.txt file EXCLUDING certain robots, or disallowing certain parts of your site from being seen by robots, you are fine. now, every search engine has different crawlers that crawl...some faster, some slower. it also depends on how new the site is, as well as if it's popular yet.
Well, i can't find anything in the admin cp that has a choice about allowing or disallowing bots, so i guess i'm ok in that area.. And are you saying that bots will index more going by the popularity of the site? Thanks for your responce matey..
Not by popularity, if you submit site map and want Google to crawl it will index whenever you update.
Ahhh, site map, that's another tricky one, iv'e heard of them, and iv'e seen something about them here, so i'll give that a go and see what happens, thanks bud..