I have a shopping site, (linked below), and no matter what I do I can't get it indexed into google. The index page is there, but nothing else. I've watched googlebot crawl a good portion of the site, at least 90 to 100 pages. It just updated last night, but still only reports the main page. Also, under a subdomain of that site I have some forums. Also only indexed the top index and nothing else. I have another forum that is configured the same way, and it's been crawled and added tons of pages to the index. It got me a 562 weight in the coop just from that forum alone. The other two add up to a grand total of 14. I'm wondering what would cause them to not include any of my internal pages. What would cause them to do that? Could I be linked in a bad neighborhood, or maybe have a few too many referneces to "online shopping" in my alt tags or something? I'm not sure what to look for, but I assume at this point it's something on the site itself they don't like.
Do you repeat the same meta data on every internal page? Do you have a static links page? Another suggestion is get some links via the coop to internal pages.
The site is dynamically generated. The meta tags are not the same, but fairly close. What the do is basically append the current product or category name to the existing keywords string and descriptiong in the meta tags. Now that I think of it, that could be a problem. The links page is accessed via a query string, so I guess that doesn't qualify as a static page, although I know it's been crawled. As far as the coop goes, I have coop ads on pretty much all of my pages as it's in the general footer template of the site. I even have a blogger blog where I do stuff and have direct links to both product and category level pages and I know it's been crawled as well since it's in the index.
I would get rid of this in all of your page title: Online Shopping @ Mari-Len.com Use that space for more descriptive title tag. Use the keyword tool; and look up skirts; for instance. For an example: change this: Prague Brown/Black/White Wool Skirt to Prague Wool Skirt or Wool Skirts - Prague wool skirt I wouldn't use anything but text letters and a dash in a title tag. If you can add more content - textwise too; that would be a good thing as well. The site also has a creation date of December 2004; so that could be holding you back.
Ok, it looks like I'm going to have to go through and make some changes. The title will have to get some work, and I'll have to go in and change some of the db entries that are used in the product page titles. I also want to do something with those meta tags. If gbot follows it's usual pattern I have a couple days before it comes back. We'll see what happens then. Thanks for the advice.
Your XHTML is so far off the spec, some parsers/crawlers may just bail on you (especially XML parsers). In some places it even isn't valid HTML (never mind XHTML). For example, in this line <td colspan="3" height="5px" class="leftboxfooter" id="categories-footer"> Code (markup): you use px as height units. In HTML this is *not* legal - only in CSS. J.D.
Well, I can't take the blame for that, I didn't write it. If you'll take a look at my other link below you'll notice the index page at least validates XHTML 1.0 Strict. I haven't had tome go go through all of the other pages yet so I'm not sure about them. I wrote that one myself. This cart app is pretty popular, and I know other's that use it are getting crawled so I'm not sure that's the problem. Besides, if I have to go through and redo all the code I'll just rewrite the damn thing from scratch. Which I may just do anyway if I ever have the time.
WOW! Usually when gbot hits my site I get 200 to 300 hits which sounds about right given the crawl pattern and the number of pages I have. It's there right now and just passed 500 hits! The number of pages hasn't changed by more than one or two, what could that mean?
The crawl finished yesterday afternoon. Over 900 hits! I looked in my logs and in my "user tracking" in the cart app. I was able to verify that several hundred of those hits were on my internal pages without the session strings in the way. Still, when it updated my listing it still only lists the main index of the site and nothing else. I also noticed something else. Many of the hits were on incomplete URLs. For some reason the last 2 or 3 characters were being lopped off the end of some of the page urls which returns an error page, but it's a valid page so I would think it would get indexed, but nothing there either. I'm not sure what to think of this. I've tried everything I can think of, and some stuff you guys thought of as well. There doesn't seem to be anything I can do to get indexed. If I can't get these pages into the index and start working on getting some rank in the serps I'm going to be out of business, I can't afford to spend a lot of money on advertising right now.