Spidering Time Limit

Discussion in 'Site & Server Administration' started by T0PS3O, Nov 28, 2004.

  1. #1
    I've got this amaxoop script and i want to display some amazon books related to what we sell. It works great apart from being dog slow! It is too slow for my standards so I am looking for an alternative but my question isn't about that.

    How long will googlebot stay around until it's fed up waiting and wanders off? i mean how long will it wait for a page to load? Does anyone have any experience with this, any educated guesses perhaps?

    Sometimes it loads in a second, sometimes in 28 seconds :mad: . I wouldn't expect the bot to waste 20 seconds. But does it wait regardless perhaps?
     
    T0PS3O, Nov 28, 2004 IP
  2. Michael

    Michael Raider

    Messages:
    677
    Likes Received:
    92
    Best Answers:
    0
    Trophy Points:
    150
    #2
    I don't think goolebot has a time out and simply uses the server timeout.

    - Michael
     
    Michael, Nov 29, 2004 IP
  3. T0PS3O

    T0PS3O Feel Good PLC

    Messages:
    13,219
    Likes Received:
    777
    Best Answers:
    0
    Trophy Points:
    0
    #3
    So that would buy me half a minute as far as the bot is concerned. Well, that would make it less bad. Thanks for the reply Michael.
     
    T0PS3O, Nov 29, 2004 IP