part of my site refuses to be indexed

Discussion in 'Search Engine Optimization' started by dethfire, Apr 17, 2008.

  1. #1
    dethfire, Apr 17, 2008 IP
  2. itcn

    itcn Well-Known Member

    Messages:
    795
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    118
    #2
    Not sure why it hasn't been indexed, but I can tell you 2 things right away which will help fix it:

    1) Use URL-rewrites to get rid of those long query strings in your pages. Make your URLs look like chefsuccess.com/stoneware-storage-rack/ instead of chefsuccess.com/reviews.php?do=view&productid=251. For instructions on how to do this just google "url rewriting [name of your webserver software here]"

    2) Create a sitemap in XML for your website and submit it to Google webmaster tools, Yahoo site explorer and MSN.
     
    itcn, Apr 17, 2008 IP
  3. sweetfunny

    sweetfunny Banned

    Messages:
    5,743
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yes it's obvious, all the links on that page are Nofollow so Google hits that page and stops.

    Both the Category and Latest Product Additions need the Nofollow attribute removed.

    Edit: Also all your Recipes are Nofollow as well, the link on the Navbar and all the sub-links underneath it. So if you want your recipe content to pull traffic, get rid of the Nofollow on that section as well.

    Ask at vBSEO to help you with rewrites of these sections.

    Leave your money on the fridge. ;)
     
    sweetfunny, Apr 17, 2008 IP
  4. itcn

    itcn Well-Known Member

    Messages:
    795
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    118
    #4
    Ah, good catch sweetfunny ... I didn't even glance at the nofollows. That would definitely explain the lack of indexing.
     
    itcn, Apr 17, 2008 IP
  5. dethfire

    dethfire Well-Known Member

    Messages:
    230
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    118
    #5
    I thought nofollow just stoped page rank, nofollow doesn't equal noindex does it?
     
    dethfire, Apr 17, 2008 IP
  6. sweetfunny

    sweetfunny Banned

    Messages:
    5,743
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #6
    In Google a Nofollow is like a brick wall, it doesn't follow the links or pass any authority. Your pages "could" get cached if they are linked from other sites, but as far as internally is concerned once Google hits your reviews.php page it won't go any deeper.
     
    sweetfunny, Apr 17, 2008 IP
  7. sheds

    sheds Peon

    Messages:
    825
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Thanks sweetfunny, for the information, i am also come to know about first time that nofollow is the wall of china between websites and Google.
     
    sheds, Apr 17, 2008 IP
  8. itcn

    itcn Well-Known Member

    Messages:
    795
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    118
    #8
    That is true about nofollow; but you can leave them as is and you CAN get Google to still index those pages by putting the URLs into a sitemap XML file. That is the entire point of the Sitemap - to allow search engines to index pages on your website which are otherwise hidden or not being found by search engines.
     
    itcn, Apr 18, 2008 IP
  9. sweetfunny

    sweetfunny Banned

    Messages:
    5,743
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I'd strongly advise NOT to use XML sitemaps on your site.
     
    sweetfunny, Apr 18, 2008 IP
  10. itcn

    itcn Well-Known Member

    Messages:
    795
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    118
    #10
    sweetfunny; ok, I gotta ask, in what circumstance would you *not* want to use the sitemap standard to submit sitemaps to the search engines? Using sitemaps is a basic and necessary part of SEO.
     
    itcn, Apr 18, 2008 IP
  11. sweetfunny

    sweetfunny Banned

    Messages:
    5,743
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #11
    I wouldn't use them in any situation. Fact, XML Sitemaps do not effect your search engine rankings and do not effect your Pagerank and this is stated by Google.

    So what's the good of them? People will probably say, they help you get indexed.

    However if your page is so weak that the only way it gets indexed is via a sitemap, that page is not going to be strong enough to rank to begin with so being indexed or not is a moot point. I'd rather my pages "not" get indexed artificially via a sitemap, because then i can identify site architecture and internal linking issues in the various site sub-sections and look at ways of correcting the problem.

    Getting them indexed via an XML sitemap throws all that data out the window, and makes it difficult to identify the strengths and weaknesses in your site. To me an XML sitemap is like trying to put a band-aid on a gaping wound, it's better to fix it then pretend it's ok.
     
    sweetfunny, Apr 18, 2008 IP
  12. SEOibiza

    SEOibiza Peon

    Messages:
    1,197
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #12
    thats an interesting thought. hadnt looked at it like that before.

    we use them as a matter of course. have always thought that getting errant pages indexed allows the linkjuice to flow more effectively through the site and so it can work like was supposed to, not with bits missing.

    always use html ones as well though too. any other reasons why you dont like them?
     
    SEOibiza, Apr 18, 2008 IP
  13. itcn

    itcn Well-Known Member

    Messages:
    795
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    118
    #13
    OK, I understand you a little better now. Yes, you are correct, it is always better to actually perform good work on a site (good & proper content, proper coding, etc.), than to just slap an XML sitemap on a site and submit and pray it all gets indexed.

    I think you may have misunderstood the purpose of a sitemap though. It has nothing to do with pagerank or search results, you are correct; but it is another tool to assist search engines in finding your pages. Sitemap.org is an emerging standard which has been adopted by the major players, especially Google, because search engines like it and are asking for it.

    If Google wholeheartedly supports it, and is asking that everyone use it, why shoot yourself in the foot and not submit one? Check out their exhaustive section on why Google loves sitemaps: http://googlewebmastercentral.blogspot.com/search/label/sitemaps
     
    itcn, Apr 18, 2008 IP
  14. SEOibiza

    SEOibiza Peon

    Messages:
    1,197
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #14
    hehe with the greatest of respect, @ 12 posts in you prob dont wanna be trying to tell sweetfunny very much just yet ;)

    misunderstood lol.

    edit and when I say lol, I mean actually, laughed out loud and spat my coffee
     
    SEOibiza, Apr 18, 2008 IP
  15. ExtraArm

    ExtraArm Peon

    Messages:
    72
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #15
    Sitemap isn't going to help if there's no PageRank flowing to those pages. Remove the nofollows, add a few more backlinks = problem solved.
     
    ExtraArm, Apr 18, 2008 IP
  16. sweetfunny

    sweetfunny Banned

    Messages:
    5,743
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #16
    It doesn't work like that, getting pages indexed via sitemap doesn't do anything to get link juice flowing to other pages. You still have architecture issues between home page "A" and sub-page "B" but all you have done with a sitemap is obscure that because subpage "B" is indexed giving the impression the flow of authority through your site is ok when it is not.

    HTML sitemaps, yes they are excellent. They provide a good navigational tool to visitors "and" search engines, they create anchor rich text links to your subpages and help create a flat site architecture.

    Nope i'm not misunderstanding an XML sitemap at all.
     
    sweetfunny, Apr 18, 2008 IP
  17. itcn

    itcn Well-Known Member

    Messages:
    795
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    118
    #17

    Admittedly, I am new to this forum, but I founded my first marketing company in 1998. I have been developing for the web since before Netscape released the 1.2 tables update (for the kids, thats when tables were invented). I have written a treatise on Database Normalization which is currently on the recommended reading list at Harvard.

    Granted, I only recently decided to come and visit Digital Point forum, but I have been working with sitemaps and google for more than a few years; trust me, I am not new to this.

    EDIT- let me add I don't mean disrespect to anyone, just fostering a healthy and lively debate and discussion on the topic.
     
    itcn, Apr 18, 2008 IP
  18. SEOibiza

    SEOibiza Peon

    Messages:
    1,197
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #18
    i know, i was just wondering if you'd seen that sweetfunny's site ranks in the top 10 for " SEO " ? :) ..and teasing a bit too if im honest. sorry ;)
     
    SEOibiza, Apr 18, 2008 IP
  19. sweetfunny

    sweetfunny Banned

    Messages:
    5,743
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #19
    Agreed post count means nothing, anyhow you seem like a smart person so what do you feel about my statement on the usage of XML sitemaps Vs Data analysis of non-indexed pages to optimize site architecture?

    BTW what was your company, there weren't many of us SEO's around back then?
     
    sweetfunny, Apr 18, 2008 IP
  20. itcn

    itcn Well-Known Member

    Messages:
    795
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    118
    #20
    SEOibiza - no worries, I was not trying to detract from anyone's ability or standing. I have no doubt anyone who is here answering questions is a professional with some experience, and many know more than I do certainly. But that doesn't mean everyone's word need be taken as gospel, there is always room for questioning :)

    sweetfunny - I mentioned in my earlier post that you are absolutely correct, there is no substitution for good, solid SEO work. Every page should be optimized as much as possible and should have the ability to be located on its own merit by the search engines.

    BUT, imagine you're working on a site with 10,000+ dynamically drawn pages (perhaps not unlike this forum). You can automate a lot of on-page SEO factors in such an environment, but it can only help to assist the search engine find all those pages by submitting a dynamically generated sitemap. Is that going to help pagerank and SEO? Not one bit, but it will help the search engines find those pages. After all, 10,000 internal site links + 1 sitemap is better than just 10,000 internal site links.

    FYI - my company is ITCN, and in the beginning we were just traditional marketing + web development. Admittedly late, we didn't begin to offer SEO services until 2002, when the dot com bubble burst and all our holdings tanked, and we needed to branch into new markets.
     
    itcn, Apr 18, 2008 IP