increasing crawling or indexing rate

Discussion in 'Search Engine Optimization' started by mdrobiul, Jul 12, 2009.

  1. #1
    I have a new site where I have more than 1 million pages. Google indexed only 300 pages in a month. I can't imagine when my full site will be finished for indexing. How can I increase this crawl rate ? I changed crawl rate few weeks ago and selected the max speed. i'm not sure but any technique you know ?
     
    mdrobiul, Jul 12, 2009 IP
  2. Professional Dude

    Professional Dude Prominent Member

    Messages:
    6,261
    Likes Received:
    430
    Best Answers:
    0
    Trophy Points:
    330
    #2
    Just try on getting more backlinks to your inner pages, and submit a sitemap.
     
    Professional Dude, Jul 12, 2009 IP
  3. atniz

    atniz Well-Known Member

    Messages:
    1,661
    Likes Received:
    28
    Best Answers:
    0
    Trophy Points:
    115
    #3
    Wow! 1 million pages on your site? That is awesome!
     
    atniz, Jul 12, 2009 IP
  4. 5kongsoft.com

    5kongsoft.com Peon

    Messages:
    26
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    all the pages are valuable or not?
     
    5kongsoft.com, Jul 12, 2009 IP
  5. mdrobiul

    mdrobiul Peon

    Messages:
    186
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    yeah. this youtube api based video site.
     
    mdrobiul, Jul 12, 2009 IP
  6. 2drama

    2drama Peon

    Messages:
    23
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    that a large number of pages

    wow
     
    2drama, Jul 12, 2009 IP
  7. earning-homes

    earning-homes Peon

    Messages:
    76
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    great number of pages....wooowww
    google index all, for some more backlinks
     
    earning-homes, Jul 12, 2009 IP
  8. vinnu

    vinnu Peon

    Messages:
    36
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    You will have to analysis your site again and generate the site map..If your content is not original then Google can avoid...
     
    vinnu, Jul 13, 2009 IP
  9. Jacob Taylor

    Jacob Taylor Peon

    Messages:
    36
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Great 1 million Pages . hmmmm great. submit Xml Sitemap
     
    Jacob Taylor, Jul 13, 2009 IP
  10. Eastern

    Eastern Banned

    Messages:
    77
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    create sitemap in xml format and submit to google webmaster tool
     
    Eastern, Jul 13, 2009 IP
  11. neiljhonson

    neiljhonson Peon

    Messages:
    315
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #11
    Submit google xml sitemap and wait to get indexed in google.
     
    neiljhonson, Jul 13, 2009 IP
  12. nidhivishwas

    nidhivishwas Peon

    Messages:
    68
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Directory submission takes less time than other activities and it is the best way to increase crawling or indexing rate
     
    nidhivishwas, Jul 13, 2009 IP
  13. theseomonk

    theseomonk Peon

    Messages:
    409
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #13
    wow 1 million pages good i think try to collect some quality back links for your internal pages and make plane for rotate every page in every section and xml site map also good idea!!!
     
    theseomonk, Jul 13, 2009 IP
  14. stevepat

    stevepat Peon

    Messages:
    316
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #14
    "I have a new site where I have more than 1 million pages" i cant really belive this , do you need these many pages ?.. I thing with 20+ u can reach on top pages in any SE ;)
     
    stevepat, Jul 13, 2009 IP
  15. mdrobiul

    mdrobiul Peon

    Messages:
    186
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #15
    ok. guys don't worry about my business. But i'm worried that if I'm going to earn. i submitted a sitemap of 40000 urls. google is indexing everyday 10-20pages. in my another site google webmaster tools shows that google has crawled maximum 6,500 pages per day. I'm thinking how to reach such speed of 6000 p/day crawling speed.

    300 pages is bringing 15-20 visitors per day from google from first page with some keywords. So 1 million pages are supposed to bring 66,666 or 50,000 per day from google. If I get 33,333 visitors per day, I'm satisfied.
     
    mdrobiul, Jul 13, 2009 IP
  16. mdrobiul

    mdrobiul Peon

    Messages:
    186
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #16
    it's not impossible that your 1 page site is getting million of visitors per day where 1 million pages are getting 100 visitors per day. content is important.
     
    mdrobiul, Jul 13, 2009 IP