How often does your directory get spidered..

Discussion in 'Directories' started by ezimedia, Aug 13, 2007.

0
  1. #1
    HI

    How often does your directory get spidered..

    I know.........

    Google
    Yahoo
    Msn

    and other search engine... spiders crawl my directory... daily

    How about you...

    How often do you get Spidered...

    Thanks
    Tom Dahne
     
    ezimedia, Aug 13, 2007 IP
  2. uttoransen

    uttoransen Prominent Member

    Messages:
    8,227
    Likes Received:
    550
    Best Answers:
    0
    Trophy Points:
    320
    #2
    google almost stays on my directory whole day long! i have selected the fast crawl option! cheers:)
     
    uttoransen, Aug 14, 2007 IP
  3. sundaybrew

    sundaybrew Numerati

    Messages:
    7,294
    Likes Received:
    1,260
    Best Answers:
    0
    Trophy Points:
    560
    #3
    I get spidered a lot,

    If I counted I would have no time to work :)
     
    sundaybrew, Aug 14, 2007 IP
  4. tolra

    tolra Active Member

    Messages:
    515
    Likes Received:
    36
    Best Answers:
    1
    Trophy Points:
    80
    #4
    I don't think the spiders ever really leave.
     
    tolra, Aug 14, 2007 IP
  5. mikey1090

    mikey1090 Moderator Staff

    Messages:
    15,869
    Likes Received:
    1,055
    Best Answers:
    0
    Trophy Points:
    445
    Digital Goods:
    2
    #5
    i think most spiders crawl a directory with backlinks every hour or so. I have a forum where there is always spiders there, they never leave. As for my directory, it gets spidered several times a day, re-cached if there is any changes to the page, otherwise it gets cached every 3-6 days. I think thats average for most newish directories.

    Dont worry about how often you get spidered. A sig link in DP will send the bots to your site all the time:)
     
    mikey1090, Aug 14, 2007 IP
  6. formula

    formula Well-Known Member

    Messages:
    995
    Likes Received:
    53
    Best Answers:
    0
    Trophy Points:
    100
    Digital Goods:
    1
    #6
    every day with mine
     
    formula, Aug 14, 2007 IP
  7. SearchBuster

    SearchBuster Peon

    Messages:
    467
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Everyday, as long as the content keeps coming the spiders will too.
     
    SearchBuster, Aug 14, 2007 IP
  8. abhishekdaaga

    abhishekdaaga Guest

    Best Answers:
    0
    #8
    I think my www.sharengain.com which is quiet a old dir... gets spidered once a day by google & other search engines crawlers also visit but never counted.... & my New Directory gets crawled very less times.... may be twice a week.... i think hey do it on a seniority basis...
     
    abhishekdaaga, Aug 14, 2007 IP
  9. abhishekdaaga

    abhishekdaaga Guest

    Best Answers:
    0
    #9
    What is this fast crawl option ??? & how to select it ??? or any trick ??
     
    abhishekdaaga, Aug 14, 2007 IP
  10. mikey1090

    mikey1090 Moderator Staff

    Messages:
    15,869
    Likes Received:
    1,055
    Best Answers:
    0
    Trophy Points:
    445
    Digital Goods:
    2
    #10
    In google webmaster tools it gives you the ability to choose how often google crawls your site.
     
    mikey1090, Aug 14, 2007 IP
  11. wamchu

    wamchu Peon

    Messages:
    2,133
    Likes Received:
    87
    Best Answers:
    0
    Trophy Points:
    0
    #11
    Its there most of the time. Few hundred time a day.
     
    wamchu, Aug 14, 2007 IP
  12. Fastian

    Fastian Peon

    Messages:
    2,085
    Likes Received:
    235
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Interesting.
    I don't see that option (for Fast) Enabled in my Sitemap Panel. I am on "Normal".

    Any tips to turn that ON ?? :)
     
    Fastian, Aug 14, 2007 IP