I've published my website with 10k+ pages in mid-December and was getting ~300 GoogleBot hits per day until recently. Now suddenly I hardly get 10. It was a sharp drop rather than gradual decline. Can it signify some kind of ban? At about the same time number of results for site:mysite.com dropped from 4k to 2k
for a site of that volume normally there would be a daily bot visit of several thousands !! however if you publish at once a large number of pages, that is suspicious to G ( and others as well ) did u have a sharp decline of DAILY NEW publishings ?? or did u even stop to publish after your first run in mid december ?? if so then you have death content and there is no reason for G TO COME AGAIN UNLESS you keep publishing NEW content on daily basis.
When initially published website had ~15k pages it was adding ~30 pages per day on average ever since. every page changes daily. Though 99% of the changes are minor - text stays the same - only some numbers change.
http://validator.w3.org/check?verbose=1&uri=http://www.cb-analytics.com/ that is a MOST serious situation - killing most likely the parsing of your site by bots and G software. normally it means that a bot can NOT parse the text at all - ( parse simplified=read/understand/processed ) in most cases. this is a different situation from just having errors and typically that error message WITHOUT errors listed means that you most likely have a mess in char set codes usually you may have one SINGLE STRANGE CHARACTER IN A PAGE causing such error msg. did you COPY and PASTE ANY of your text into your pages ? did u use MORE than ONE editor to CREATE your text ? do you have your offline editors PRECISELY configured to the charset of your online HTML pages ?? in addition to that you have the meta tag for your charset definition MISSING !! make sure you add the SAME charset definition than the one configured in your editors used to CREATE or edit your texts! in modern times UTF8 BOTH online and offline might be a good option - BUT you NEVER can just change your charset - when switching from ONE charset to another you have to CONVERT your existing files into the NEW charset !! also if active publisher you may really want to create your own RSS feed - either by a suitable SW or manually ( I have 2 manual and 3 automated feeds ), register your feed with all RSS directories, then ping all after each new2 article / page published. that invites major SE since all new feeds appear within minutes in all directories pinged. RSS directories are most active and thus get plenty of major SE bot visits. place an sitemapindex.xml file into your site and add the URL to robots.txt active publishers with such volume site usually get many thousand G bot visits/day. just WELCOME/invite the bots by all modern / legal means as a.m.
hans, i see you do a good job... before i will forward this last message to my programmer, him to look at it, i would kindly ask you to revise, in the same mode my site too: http://www.cigbuynow.com. I guess there is also a mess there, cause i received a penalty or something for i do know what and dropped for -100 positions on by my main keyword. thank you in advance
Fixed the encoding problem - I had the offending character on the website from day 1 so I don't think it was the problem. There are bunch of other validation errors. I'll need to take a look at them. Can this really mess up crawling that bad?
similar but different see http://validator.w3.org/check?verbose=1&uri=http://www.cigbuynow.com/ example you have a meta tag outside/before your actual <head> sections starts - see <link rel="stylesheet" type="text/css" href="styler.css" /> the / YOUR problem is that someone ELSE is doing your site ... I started nearly 11 yrs ago - stated with ONE page ( as opposed to your thousands ) and I allowed/forced myself to grow WITH my site. doing all on my own. step by step. true success is the result. thus if you are serious about being a true professional web publisher take your time allow yourself to start smaller and grow slower but steady and learn as needed whatever needed toi have full control of quality / SEO / adsense optimization as well. many ten k$7yr MORE revenue is the reward for harder/more work from your side. my own experience has shown that typical coders / programmers have little or NO knowledge in SEO and HTML and mostly focus on optical appearance of sites only. how could he otherwise ... because publishing FULLY optimized means investing thousands of hrs / yr in learning and optimizing - your salary FOR your programmer thus would have to be in th e6 digit$-range/yr to create the motivation needed for a perfect job. only the site OWNER - earniing a LIFETIME pension out of your own site could have the motivation to do ALL the work needed for perfect web publishing jobs. starting a huge site like yours, look at it as your LIFETIME pension scheme - for YOUR old age retirement years as well as for NOW. your site is your life and supporting your future. re ur: Can this really mess up crawling that bad? I would NEVER waste my time HERE if it couldn't . it depends on TYPE of error. but i have no intentions of analyzing erorrs. a publishers goal is to have clean validated code. else no guaranteed success.
Haven't started with these yet. I've been developing the website since December and getting some targeted traffic from forums. I'm starting to focus on SERP ranking only now. I guess using Google Webmaster tools will not hurt. Does it help much?