1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Problems with Inktomi Crawler and Metatraffic

Discussion in 'All Other Search Engines' started by TampaDave, Mar 30, 2004.

  1. #1
    Metasun.com makes a good stats package you can run off Access or SQL Server. I'd been using it for some time when, at the beginning of March, I noticed the Inktomi bot was getting 500 errors when it tried to crawl my site.

    Much back and forth between myself and MarketLeap (I paid for inclusion). No resolution.

    By use of a free page optimizer, I found out some bots weren't seeing my <body> tag. If I took out the Metatraffic include, the page optimizer saw it.

    So I left it out and two days later, Inktomi indexed the pages no problem.

    More experimentation revealed that the lost <body> tag only occured when using the #include or Server.Execute command. Using the javascript logging method didn't seem to cause a problem.

    Just a caution to those of you who might be using Metatraffic as your stats package.
    SEMrush
    TampaDave
     
    TampaDave, Mar 30, 2004 IP
    SEMrush
  2. hans

    hans Well-Known Member

    Messages:
    2,924
    Likes Received:
    126
    Best Answers:
    1
    Trophy Points:
    173
    #2
    to my knowledge the Inkotmi bot is slurp
    21 2772 0.56% Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/
    that is the slurp visit this month - for free :) inclusion

    i have seen no slurp problems exept that slurp seems NOT to process a 404 and dump that page - but slurp comes back for YEARS to missing files ..

    googlebot had about 9000 hits ( theoretically each file 3 x a months ) in march
     
    hans, Mar 30, 2004 IP
  3. TampaDave

    TampaDave Peon

    Messages:
    14
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yah-

    Slurp is free, but when you launch a new site, you can pay your $39 and get listed in 48 hours instead of 4 weeks to 6 months. I only paid for home page inclusion.

    Google didn't seem to have anyproblems indexing even when using the #include or Server.Execute

    TampaDave
     
    TampaDave, Mar 30, 2004 IP