1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Increase Website Crawl Rate

Discussion in 'WordPress' started by mavrincgreen, Feb 19, 2012.

  1. #1
    Hello, I have got a new domain & i do posting regularly but my post are not crawled regularly. I wanted to know how can i improve my crawl rate ?
     
    mavrincgreen, Feb 19, 2012 IP
  2. waziuz

    waziuz Active Member

    Messages:
    783
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    80
    #2
    From my own experience, the crawl rate increases with the increase in website authority and site age. And make sure to use sitemaps and submit it to Google webmaster tools. Also check inside webmaster tools if there are any problems in robots.txt
     
    waziuz, Feb 20, 2012 IP
  3. Rajnish357

    Rajnish357 Well-Known Member

    Messages:
    157
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    103
    #3
    Its easy, Go To Webmaster Tools> Site Configuration> Settings> And drag th Crawl Rate Slider to maximum and save. :)
     
    Rajnish357, Feb 20, 2012 IP
  4. explainstuff

    explainstuff Peon

    Messages:
    49
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Doing this, will sure increase the crawl rate.... but at the same time it increases the bandwidth usage with the crawl attempts. If you don't have anything new.... every few minutes.. it doesn't really make sense to drag the crawl slider to maximum.....

    Optimally, you only need a bot to go through your website when you have changes...... Post or add content to your website almost at the same time every day. This lets the bots know what time frames and frequencies you are usually making posts... this will also decrease the usage of your bandwidth.... This method has also been mentioned on google forums...

    One other thing is, if you make posts or add content let's say every day.. then you could also add the meta tag.. visit every day....Google it .. you'll know what to do :D

     
    explainstuff, Feb 20, 2012 IP
  5. Icecube_media

    Icecube_media Peon

    Messages:
    656
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    HI
    yes as all say, sitemap help you for getting crawled by the search engine.
     
    Icecube_media, Feb 20, 2012 IP
  6. Rajnish357

    Rajnish357 Well-Known Member

    Messages:
    157
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    103
    #6
    Sitemap is good Idea but it will only help when robots come to your site quickly !!, If robots are coming in every 48 hours then site map is not going to do anything alone..
     
    Rajnish357, Feb 29, 2012 IP
  7. mkpoway

    mkpoway Peon

    Messages:
    6
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Start building backlinks, it will make it more likely that the Googlebot will visit and crawl your site.
     
    mkpoway, Feb 29, 2012 IP