to all those who have a problem in either getting vlsited or getting all pages listed a few tips: 1. different bots react differently on HTML errors remember that a bot is basically very much like the SW of a browser, except that a BOT is tine, hence a bot NEVER has any error compensation built in, while a browser has more or a lot of code error compensation built in. as a result a page with LOTS of code errors still MAY display correctly in a browser, but a bot may struggle at a PARTICULAR / bot specific fatal code error and the indexing may go UP TO that fatal code error and END there, resulting in a partial indexed page even if the cached version LOOKS fine, the parsing of the CODE vs the parsing/extraction of CONTENT / a lot of tags / image tags, links, image labels, anchor text, etc ... MAY be either missing iin the parsing of indexing process or only partially processed. due to code errors. example: A. if a tag is open and the closing tag missing, then a bot and the parsing SW after MAY consider ALL part after the NON-closed tag as TAG and NOT as content. B. if charset definition ( line 1 ( ONE) of each document, is missing, a bot and all the parsing SW never ever could KNOW what code follows, looking at global content in cyrilic, latin charsets, utf8, chinese, etc - ONE byte may mean a thousand different things UNLESS clearly defined by a required charset definition!!! the typical fallback default may be utf8 for typical SE - but if that undefined document is anything else like iso8859-3 or watever windows or asian or other language/charset ... then NO proper processing / parsing is possible - yet those pages still appear properly displayed in the cached version of the SE: but the fact that an original/correct looking page appears in the SE cache in NO way proves that those mis-configured or UN-configured/UN-defined data have been processed/parsed at all or correctly!!! 2. learn to SPOIL bots and SE by offering PERFECT and fully validated pages. treat SE like your lover girl - offer all help in the cleanest possible way: if for any reason you wish no sitemap, ( i offer NO sitemap for a simple reason as it has been proven after repeated extensive tests to be absolutely useless/worthless to my large site) - but at least you can offer your RSS feed in the robots.txt as recently specified. this robots.txt submission version is new and may be unknown to many. offering a correct link to ALL your RSS feeds which in turn contains all your most recent articles posted, will do the same as a sitemap, except that it is used by all major bots. 3. of course nowadays each web site should offer at least ONE RSS feed, or if topics manifold - then one RSS feed for each sub-section of a site. these RSS feeds NEED to be registered by all major RSS directories AND those RSS directories also need to be PINGED after each new RSS item being posted. with this RSS feed publishing procedure your feeds - hence your newest article-links will nearly instantly appear on all RSS feed directory sites as well as on the sites offering YOUR feed as SSI content on their pages. hence all major bots will find your newest pages much faster, and since you will have approximately50+ high value / high PR backlinks from RSS directories or blog directories, your site will be crawled faster and more thoroughly than simple neglect hobby-sites. 4. and finally as posted in various other threads, submit your site and/or major pages to MINOR but OLD SE - that gives you backlinks/PR but more importantly more reputable links BACK to your site for the major SE to find your site / major pages. at least a dozen or so minor SE should be used for fastest publishing success. its free, its efficient and helps you a lot for a little extra efforts. needless to add that a site needs new fresh to be attractive for major SE. be dynamic, creative and PRODUCTIVE if you want to be spoiled / loved by any and/or all major SEs. before whining and blaming SE, make sure you as webmaster have mastered your product and all is in perfect shape and condition presented to the world of bots. avoid whining as long as one single above point is either incomplete, missing or WRONG. most of the sites visited in such threads have very substantial basic errors - use all available professional toosl to verify, validate all aspects of your site validate robots.txt RSS feeds CSS HTML code each of above has a specialized validation service online to be used for free by all who love to be efficient AND successful
Really? I disagree. You can spam the MSN search engine, and people do it - and search results are usually somewhat unrelated to what your searching for.
easy to know/figure out you load the page from where you link in your browser - source code view mode - what you see there is what a bot sees. if you validate that page - 100 % validated - then a bot finds valid code including valid links. landing page = same procedure. a bot sees whatever code a server serves - NO javascript, NO java, NO flash - plain HTML in a validated version. when you use online validators , they see exactly what your server serves - hence erorr or valid. there is NO "little error" or "few error" there is NOT valid valid like there is NO little alive or little death either alive or death! there might be ONE single error causing a termination of parsing text. example about 2-3 yrs ago was a site owner asking for help ( in DP ) a review showed a LINE break in the test of metatag <title> that cause googlebot to END correct parsing after that linebreak early in the page = result he dropped by many ranks for the keywords of that page. after problems resolve all back to normal. the issue was that this kind of error is NEVER detected ba HTML code validation - only by visual direct inspection of SE RESULTS shown and source code compared.
MSN is always unrelated to what I was searching for. Google, and I would go as far as saying yahoo as well, are my more intuitive.
last month MSN/live had almost double the bot activities than G as a result this months the MSN traffic goes really UP and since the possible deal between Y and MSN cracked in recent weeks - Y as well substantially increased efforts and thus brings more and more traffic these past 2-3 weeks. both are nearly head to head in the race for being nr 2 SE - but dimensions away from G. nevertheless a few k uniques more from here and there also boils up to hundreds of k monthly visitors. both MSN and Y increased the last few days even more - G even MUCH more during same days
MSN indexed my sites much more faster then yahoo or google ever did. Though yahoo now spams my sites with 30-40 visits per day from their bots. As an user, I agree to one post above that MSN search results are totally different of what I'm looking for.
30-40 slurp visits / d certainly is NO spam at all unless you have only 2.3 pages Gbot normally revisits entire site ab out 6-10 times per months to assure all searches are fresh and up to date or if you have 1000 pages - Gbot comes about 6000-10'000 times per months that is NO spam that is efficiency
Most of my traffic comes from MSN, about 20% from google, and not one stinkin' person has come from yahoo. I'd like to know the secret to getting any yahoo listings myself.
After reading this thread, I decided to fix the HTML validation errors that I've ignored for the past few months.. I used to have page 1 rankings for a TON of our keywords, and then when Live made their algo changes, we disappeared off the face of the planet. I don't know if their bots were stumbling on some of the errors that were previously there, but something was wrong - Out of over 40K pages, they had only indexed 1500, while Google and Yahoo had picked up nearly all of the pages. Everything is now 100% XHTML 1.0 compliant, so now it's just a waiting game to see if anything changes as a result of it..
I think the problem (for me at least) is not related to coding errors. Using out of the box CMS's. Like wordpress. Sites rank fine in Y and G. Not indexed in M. Just a lame SE IMO.
You can submit your site here: http://search.msn.com/docs/submit.aspx Thats it I think. They basically tell you to sit back and let them pick everything up and for you not tow worry about it, however google is indexing my pages within a day, and MSN has yet to index anything BUT my homepage. All I can say is OMG!
Hi, I've had the same problems with MSN <--> Live search regarding listings. There are a few new things on MSN Live that could help: 1) use +link:domainname Example for my sample site: +link:www.materialwords.com When people thought link:domain was out. MSN announced that its in with +link: and +linkdomain. Try this and find sites linking to your site on MSN. 2) Use Sitemaps on MSN. AARGH GOOD NEWS, but HOWWWWWWW ???? Simple: use robots.txt. The first line on robots.txt could list the sitemap on your site WHICH according to rules must be in the topmost (i.e. root of your website directory). (see: sitemaps.org or Google robots.txt ) I tried this and my listings improved on MSN. MSN attracts great traffic. Windows and IE search defaults use MSN. Good Potential Traffic. Hope you enjoy your time with MSN. Don't FORGET to SUBMIT. Cheers Rehan
How can you list the sitemap in the first line of your robots txt file? I don't understand. Other than that, I have done everything to get on MSN.