hello all, I would like to consult with you on how to deal with unindexed pages of a website in G. for a website i promote, it has 300 pages, 120 of them are not indexed. the site is online for over a year. i've done the following: internal links onsite to these pages external links leading to these pages up-to-date sitemaps -xml,php and on G webmaster tools meta tags+robots.txt - follow , index. the only thing i belive should be done, is to update those page's content more frequently, (i check on some of them - last crawl was 27.4.09). so, how do you guys deal with this?
It's just the way Google operates. I would use Google Webmaster Tools to analyze and fix your site (Dup content, errors, etc.), submit a complete sitemap, and see what happens.
Also try and get links to the internal pages. That should help get them indexed. Also make sure that there are no duplicated pages. Josh
i've checked on webmaster tools - not problems with any urls, only 2 restricted by robots.txt. other than that - no errors.
How many clicks does it take to get from the front page to the most buried content? Google advises everything should be kept as close to the home page as possible, 2-3 clicks at most unless your site is just massive in size & popularity. Another potential problem is if the pages don't have a lot of content Google can recognize. Is it video, audio, or images on the unindexed pages? If they're lacking a chulk of unique textual content, G won't know what to do with them or will assume they're too thin to be important.
pages are a maximum 3 clicks from homepage. korr - i also believe this could be the problem - a lot of the un-indexed pages are video\picture pages (althoug i keep adding proper alt tags to each). i think more content should be added to make these pages updated more often, but putting that aside - is there any technical thing i didn't think of?
Is your Internal link building strong? This is very important. 1)Try to get link to all your unindexed pages from an indexed page. 2)Try to submit all your unindexed pages to social bookmarking sites. Some times google feel difficulty to crawl your internal pages if your internal linkbuilding is week. 3)Make sure you dont have any html errors in unindexed pages. 4)Make sure you have submitted sitemap. thanks.
thanks the sharing,that's really very useful. but also i think maybe all of you miss some point:the content of your page!how will google consider your pages,is that really unique or useful content to its users? if google think they're duplicate or less info i dont think they'll get indexed whatever you do. any idea?
I agree with adding more content, but constantly updating what should be static pages is a bad idea because it resets the age of the page. I had an image site where the only text content in the page was details about each image and some tags someone might search for to find the image. Google originally went through the sitemap and indexed everything in hardly any time at all. Later on Google dropped thousands of pages from the index. I started adding more specific titles for the pages along with a quick paragraph about what is in the image, Google started indexing those pages as I updated them again.
My advice is to get some backlinks for the main categories of the inner pages instep to get them indexed asap.. You might also consider adding some fresh unique content and maybe submit separate XML sitemaps to Google based on each section links
i made a list of some of the un-indexed pages (which are very smilar in layout and content), and will divide them into 3 groups: 1 - i will backling + add more content. 2- only backlinks 3-only adding content now i will take a closer look on their indexing. one more thing - i've noticed to all of these pages are very similar - they small amount of text, and are composed of video or images. i will update you guys when test is completed. thanks everyone for sharing and trying to help!
120 unindexed pages is a lot. Every time I add a new page, I submit it to a variety of social bookmarking sites. That gets them picked up pretty fast. For a small number of those unindexed pages that you consider important, you can place them (4 at a time) in your Digital Point signature until Google indexes them. Then remove those four and put four more into your signature until they get indexed. Keep repeating that. Labor intensive? For sure. But it will work and will get your pages indexed.
You mentioned you had the interlinking working well, which is a good thing, but do you also have unique Meta data: titles/descriptions/tags ? because if not G will see it as a duplicate page most of the time and not index it. If you have backlinks (which sounds like you do) I'm assuming G assumes it is a duplicate either because of onpage SEO or the pages are too similar (text within the page) to be considered unique. Like Jim4767 stated, when in doubt do some social bookmarking and more link building to ensure it's all indexed and ranking properly.
Jim4767 - thats a good option thanks for the suggestion. GameFriends.com - all meta tags and content are unique, one thing that site doesn't have is duplicate content (amen to that).
How easily your pages get indexed has a lot to do with Google's trust in your website and your website's age. With that in mind, you should focus on promoting your website through whitehat means and continuing to publish fresh content.