1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Googlebot Spidering...

Discussion in 'Google' started by digitalpoint, Mar 7, 2004.

Thread Status:
Not open for further replies.
0
  1. #1
    Man, can I just say it's amazing how fast Googlebot spiders these days. Within 6 hours of this forum having it's first inbound link, Googlebot was visiting, nd pages were within the index within 48 hours. Since then, it's visited every day so the content within the index is practically realtime (with a 48 hours lag). :)

    A perfect example is a #1 ranking for free Yahoo inclusion (not all that competitive I know, but still)...

    Now, I just need to figure out how to get Googlebot to *stop* spidering do quickly and stop sucking bandwidth. I ended up adding a few entries into the robots.txt file to keep it from grabbing redundant info at least.

    - Shawn
     
    digitalpoint, Mar 7, 2004 IP
  2. Foxy

    Foxy Chief Natural Foodie

    Messages:
    1,614
    Likes Received:
    48
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Yep - well done

    I had a site that I bought on Thursday last month - it went live on the Friday and I was, like you No1 [also not a terribly competitive term] by the following Monday - and, I only found out by accident when i looked at the stats!

    The site only had 1 page as I was thinking oh well it will take a week!

    Did I scramble around then!

    Interestingly it started with a 4bot hit then the next day 6bot and then the third 2bot - I noted the total hit count for each day.
     
    Foxy, Mar 7, 2004 IP
  3. Mr T

    Mr T Guest

    Messages:
    62
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #3
    New content boost maybe? Good work though, last new pages I put up took a good week or so to appear.
     
    Mr T, Mar 8, 2004 IP
    Mikejohnson105 likes this.
  4. Foxy

    Foxy Chief Natural Foodie

    Messages:
    1,614
    Likes Received:
    48
    Best Answers:
    0
    Trophy Points:
    0
    #4
    This occurred to me but the term was very specific [but not what I had intended, hence not looking] and has remained stable since - not much else has happened up to now except googlebot visits so I will wait and see.
     
    Foxy, Mar 8, 2004 IP
  5. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #5
    I'm not really sure why Googlebot visits so often (www.digitalpoint.com is visited very often as well). And also keywords within digitalpoint.com always seem to rank high with little effort. Almost makes me think there is some sort of bonus simply for being within digitalpoint.com in Google's eyes.

    - Shawn
     
    digitalpoint, Mar 8, 2004 IP
  6. Foxy

    Foxy Chief Natural Foodie

    Messages:
    1,614
    Likes Received:
    48
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Probably as an authority site - which you are :)
     
    Foxy, Mar 8, 2004 IP
  7. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #7
    Well, at least I know if I wanted to dissolve the business, I could use the domain for driving affiliate traffic or something... {lol} j/k

    Of course, we sell actual products, so that wouldn't make much sense because we make a decent amount with those.

    - Shawn
     
    digitalpoint, Mar 8, 2004 IP
  8. GuyFromChicago

    GuyFromChicago Permanent Peon

    Messages:
    6,728
    Likes Received:
    528
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Hey everyone. I'm new to this forum. Came across this site looking for Google api stuff - great tools on this site.

    As far as Google goes, the Goolgebot is busy. It hits my site(s) almost once a day, every day. I update content quite a bit, and have bandwidth to spare. I actually like the bot to stop by often. I remember not too long ago when it would take weeks to get listed...not anymore.

    Looking forward to some good dicussion on these boards. Thanks again for the tools.
     
    GuyFromChicago, Mar 8, 2004 IP
Thread Status:
Not open for further replies.