Hello, I published my first website a couple of months ago and within a couple of weeks had a few of my pages indexed by google showing description etc. I was also showing in top three for allinanchor allintitle allintext for my three chosen keywords allthough only showing on about page 50 in SERPS and with PR0. I have now found that google now only shows my url and non of my pages seem to be indexed. I still show as #1 in allinanchor but don't figure in allintitle or allintext and don't show in SERPS at all. I am very new to all of this and was wondering if this was maybe part of the "sandbox" effect or something else? Any comments on what may have happened would be greatly appreciated. Richard
Google has been cleaning up its database, and in the act has been removing millions of 'duplicate' pages. Tens of thousands of pages have been wiped off from fakey directories that create categories with little or no content. If your site is one of these, then there is a fair chance that this has what has happened. First pages are moved to the 'suplimental index' they show a cache of 1969, then they become partially indexed (as in your case)
Hello OWG, Thank you for the quick response. Hmm I used a bit of software from spyderopts.com that basically builds "doorway" pages from my index page which I did for other keywords in my field, this software then built similar pages but optimised for these KW's and uploaded them to my server. I suspect in my ignorance this is may be what has happened. Do you think that if I remove these pages google will then re-index me or will I now be in their bad books?
OWG, I have had the same problem with some of my pages - I know they are not duplicate, but all they have is the domain name as the title tag.. any other ideas why? I'm guessing it's because they don't as such have a link from a indexed page, because they are dynamic pages, and are found via a javascript drop down menu. Would you agree? I have got around this by setting up a dynamic page from the homepage which links to all of the pages that are found via the drop down menu - do you think this would sort out this problem, and how long would it take in your opinion for these pages to have the description instead of domain title. Darren
Could you post some links to this info OWG, or perhaps elaborate on where you read about this so I can do a little reading (it fits - and I want to know more) ? You're ringing some bells with the cache thing. Cheers, JL
Wow OWG, talk about dropping a bomb shell. Does that mean that any pages showing a cached date will be dropped eventually? Where did you receive this info?
I have been watching this for a little while, and have noticed a common pattern in what happens before pages get wiped out. first they go to supplimental, they tend to show the cached date of 1969, then they go to partially indexed, then they just go. I will not name directories that I have noticed it happening to as I don't feel it is the done thing. I can't really point you to any thing to read, as it is work I have done for myself, and as I don't do the article writing thing it is not published. I would say though that go to any large directory, o a site: search on G, and you will see the pattern over the space of time. Google is really having a go at some big sprawling directories, as well as smaller ones.. Sorry I can't help out more.
On that topic.. does anyone know appx. how "different" a page must be for Google to consider it a unique page in your network? Does it just need a different title and somewhat different file name? Or is it scanning the whole thing for actual content? I ask this because I was thinking of adding a "gift ideas" thing to my site for the holidays/future gift giving occasions, and a some of the categories would obviously have some duplicate content.. just wanted to make sure I wouldn't be too harshly penalized for it.
I have seen pages that were 95%+ identical that were indexed and over a long period of time. I have never read specific numbers, but I imagine that at least a 5% variation is needed. What that variation is, who knows...
OWG, Thanks for that. I've seen the same thing myself, but never really put the full picture together until you described it. The cache date is of particular interest to me. It's a simple persistent flag I can use. Researchtechs. There's a number of things that can be assessed by humans, while only a limited subset can be assessed by engines (too much overhead). The first and most obvious. the DOM, or document object model. Basically, all the tags within a document form a pattern. The pattern is expressed mathematically with a very simple string - not unlike XML. Comparing the two is a snap, and variations are immediately apparent. Remember, we're talking about markup only, not content. Naturally, it's perfectly OK to have the same pattern - we all use templates from time to time - especially with dynamic data. But this is much more easy to represent, as it's far less data. Variations in the dom (from my experience) have been enough to keep a page fresh (in terms of engines), along with minor changes to the content. On the topic of content however, it's often a wise idea to change the amount of data within a page, and, of course, distribute it across your page (well, within the body) Amazon (API) is a handy example. I myself frequently include editorial reviews (searched at request time according to the topic of the page). The general idea is to go after a group of results, maybe ten. You then randomly select 2 or three from those ten, and distribute across your page. You must ensure it's not hidden, and there's no point, because they are on topic/theme, and remember the Amazon TOS limit lifespan of same to 24 hours. You must also be linking back to amazon from these, so people can find the books if they want to. It's a great way to pad out directory pages, or improve on the usual odp scenario too. As I think most will recognize, editorials are rich in natural keywords, and that's a definite plus from an SEO side. There's other data out there that you can get your hands on, I'm sure there's a thread, prolly around 6 months ago now, but it was all about content sources. Hope that helps, JL
Nice work JL, Someone would be wise to publish what OWG and JL have come up with here... with their permissions of course
Shawn and others have mentioned it on various threads. Most predominantly in those in the Coop network forum where people saw their site: page number drop bigtime. Turned out that most of these were indeed 'copies' or duplicates or no-content pages. Makes sense as well. More and more people just use amazon and wiki and odp data to build large size sites for link building. If it doesn't add value, it's basically spam. Google doesn't like that and I don't like it either to get 10 identical results from 10 different URLs finding they all syndicated the data frmo one source. One copy in the index is enough.
The information is out now. Those who do this for a living would have noticed it the same as I have (if they do the job right). have been meaning to write it up for weeks, but figured that just posting it on the forums would get it out there well enough to those that matter.
Well I cleaned up my pages a bit and removed duplicate pages etc. and re-submitted my URL to google, I have now been spidered twice in the last couple of days and now my pages have been indexed again showing titles and recent cache date etc. I am now waiting like a coiled spring ready to burst out of the sandbox!!