1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Errors that aren't on my site

Discussion in 'Google Sitemaps' started by Xig, May 23, 2006.

  1. #1
    I've had a site live for a few years, and enjoyed strong SERPs until yesterday. The site runs several levels deep (homepage -> category -> topic -> documents) but everything past the first two levels is no longer coming up in Google search results.

    I fired up Google Sitemaps to see what the problem might be, and sure enough it has returned lots of errors. But none of them are my fault.

    Like most sites, we track clicks on external links by linking to "www.domain.com/click/X" (where X is the link ID) and then forwarding the user onto the URL they clicked on. This is a simple "Location: domain.com" in an HTTP header with a 302 redirect.

    Although we check these links every week, some bad ones do slip through. But Google Sitemaps lists the errors under our "www.domain.com/click/X" URL rather than placing the blame on the other site...

    So I'm guessing Google have decided that a certain percentage of our pages are 404, 500, etc and that our site is junk!

    Has anyone had any experience with this? Is this a known error? Could this have caused the drop in SERPs?
    SEMrush
     
    Xig, May 23, 2006 IP
    SEMrush
  2. MaxPowers

    MaxPowers Peon

    Messages:
    261
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #2
    It could have caused the drop based on what you were saying in this post. If Google thinks your site isn't maintained well and that people clicking their serps may get an error when they visit "your site".... You'd have to be careful with your redirect script to be sure that it is sending proper HTTP headers to Google or other crawlers.

    Another option is to leave the links as natural outbound links and use the Ajax Link Tracker from http://www.glennjones.net/Home/ to track your outbound links. This tool catches the outbound click and uses Ajax to populate your database while the surfer is loading the new, external page.
     
    MaxPowers, May 23, 2006 IP
  3. Xig

    Xig Peon

    Messages:
    75
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3
    This is what worries me. I can't understand why Google would penalise my site for errors on others. Maybe external links to broken sites might affect my PR a little bit, it shouldn't count them as errors on my domain and penalise me.

    These are the headers I receive when I request the redirection page:


    Thanks for the tip, I'll knock up a demo and see how well it reacts. I'm always a fan of doing things server-side if it can be done though. Client-side never seems as reliable.
     
    Xig, May 23, 2006 IP
  4. MikeSwede

    MikeSwede Peon

    Messages:
    601
    Likes Received:
    16
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I don't think it has anything to with the errors that your site only has the first 2 levels showing ut. Google is changing, whatever they think they need to fix, and it has "deleted" a lot of pages from their index.
    We can all see errors like this and there is nothing you can do about it. It is not something with our sites, it is just that Google is all f**ed up
     
    MikeSwede, May 26, 2006 IP
  5. MaxPowers

    MaxPowers Peon

    Messages:
    261
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #5
    With a sitemap it's possible to keep all of your pages within two clicks.... one to the sitemap, one to the next page.

    The old sitemap adage goes "every page links to your sitemap, and your sitemap links to every page". This means they are 2 clicks from each other.

    According to rfc code (seriously paraphrased), the 302 tells Google to spider the same URL later to see where it goes. A 301 says anytime you see this URL, think of the one I'm about to give you. I'll stop here lest this become a 301/302 debate

    Google isn't penalizing you fo rtheir site, it's penalizing you because you have the broken link. AutoMapIt does link-checking and sitemaps with the upgraded service, or Xenu LinkSleuth is an awesome link-check software for free. Either way, keep up on your links to avoid hosting broken ones.
     
    MaxPowers, May 26, 2006 IP
  6. Roman

    Roman Buffalo Tamerâ„¢

    Messages:
    6,217
    Likes Received:
    592
    Best Answers:
    0
    Trophy Points:
    310
    #6
    Google is broken, search the forum and you'll find many threads about it like this one. It has nothing to do with your site.

    If you had strong SERPs til yesterday I highly recommend you due you due diligence before taking any advice and making drastic changes to your site.
     
    Roman, May 26, 2006 IP
  7. Xig

    Xig Peon

    Messages:
    75
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Thanks everyone for your advice. Broken links can be tough when there are so many URLs to check - I have a server dedicated to checking them and the cycle takes about 2 weeks. It seems 2 weeks isn't fast enough to catch them when they break.

    I've rewritten the sitemaps script to ping Google nightly with fresh content, and temporarily added "nofollow" to the external links and put the redirect pages in my robots.txt exclude rules. Hopefully this hack will tide me over until Google unf*cks itself.

    I was tempted to make major changes, but I agree with you Roman - I shall ride out the storm and see where I am when everything returns back to "normal".
     
    Xig, May 28, 2006 IP
  8. Xig

    Xig Peon

    Messages:
    75
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Just a quick update on the issue. The situation seems to be getting worse - a ton of old URLs from January 2005 suddenly appeared as supplemental results, hung for a few days, and departed again.

    GoogleBot is crawling the site like crazy, but I haven't seen anything show up in the index yet. Hopefully good things are on the horizon.
     
    Xig, Jun 1, 2006 IP