1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

15K+ pages, how to be crawled fast?

Discussion in 'Google' started by tolgafiratoglu, Dec 6, 2008.

  1. #1
    Hi there.
    I have a project with more than 15K+ pages, Google didn't crawled all yet.
    It's PR0.

    What should I do to be crawled in 3-4 days, or is it inevitable to wait a month?

    In blogs blog-ping gets the content in 1 day, but in photo archieves, I'm not sure.
    SEMrush
    Tolga
     
    tolgafiratoglu, Dec 6, 2008 IP
    SEMrush
  2. digidownload

    digidownload Banned

    Messages:
    13
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I think you should try using some link spamming softs. Thats the only way to get so many pages indexed so fast.
     
    digidownload, Dec 6, 2008 IP
  3. Reprobate

    Reprobate Guest

    Messages:
    1,060
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    90
    #3
    What is the content like in these pages?

    That's a LOT of pages for a search engine to consider. And I'm not talking about the amount of time it would take to crawl, but the quality of said pages.
     
    Reprobate, Dec 6, 2008 IP
  4. McDonalds|Chicken|Selects

    McDonalds|Chicken|Selects Peon

    Messages:
    58
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Ping and add a sitemap
     
  5. tolgafiratoglu

    tolgafiratoglu Active Member

    Messages:
    249
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #5
    Thanks for your offer, I searched read and learned how to ping a site-map.
     
    tolgafiratoglu, Dec 6, 2008 IP
  6. kappaknight

    kappaknight Peon

    Messages:
    160
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #6
    I would definitely create a sitemap and submit it to Google's webmaster tools. However, even with that, expect at least a couple of weeks before things really pick up.

    You're also not going to get 100% index rate instantly; expect about 30-40%.
     
    kappaknight, Dec 6, 2008 IP
  7. Shellerz

    Shellerz Active Member

    Messages:
    1,011
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    78
    #7
    With that vast amount of pages you need to be indexed, urely a quick sitemap to send google bot to most important pages will give you a very positivie start in terms of indexing :)
     
    Shellerz, Dec 6, 2008 IP
  8. IEmailer.com

    IEmailer.com Well-Known Member

    Messages:
    1,864
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    110
    #8
    Create a separate XML sitemap for every main category and then create a master XML sitemap for the main categories then submit all the XML sitemaps in Google webmasters tools, under your website account..

    Get more quality links from high PR pages, and also try to get inner links to the main categories..

    After all, the quality of the content in those pages might become an important factor..

    If you think that some of those pages are with no value, try not to force the search engine to index them as they won't help your website ranking at the end ;)
     
    IEmailer.com, Dec 6, 2008 IP
  9. cheapez

    cheapez Active Member

    Messages:
    1,123
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    78
    #9
    There is no way to crawls all 15k pages in 3-4 days. It takes months, and even so not all of them will get indexed.
    Submit Google Sitemap will helps, get more people link to your site and post new contents on your site.
     
    cheapez, Dec 6, 2008 IP
  10. lifeplayer

    lifeplayer Notable Member

    Messages:
    15,612
    Likes Received:
    475
    Best Answers:
    0
    Trophy Points:
    265
    #10
    Seems it is impossible to get it done by few days, it might take a month to get done.
    However, try to use sitemap, it can helps to get it done 10~20% within 3~4 days
     
    lifeplayer, Dec 6, 2008 IP
  11. tendulkar2

    tendulkar2 Banned

    Messages:
    2,618
    Likes Received:
    69
    Best Answers:
    0
    Trophy Points:
    0
    #11
    I think it takes lots more time than that... I fastest way I know is Submitting a sitemap to google webmaster tools.
     
    tendulkar2, Dec 6, 2008 IP
  12. axlarry

    axlarry Notable Member

    Messages:
    3,961
    Likes Received:
    121
    Best Answers:
    0
    Trophy Points:
    240
    #12
    Though I doubt google will index all of them in just 3-4 days, you should try this :
    Make rss feed and ping it, make a sitemap and submit it, do some link building, and make sure no duplicate meta tags on each page of the site.
    One more thing, you do have unique content right?
     
    axlarry, Dec 6, 2008 IP
    web-fanatic likes this.
  13. web-fanatic

    web-fanatic Banned

    Messages:
    727
    Likes Received:
    170
    Best Answers:
    0
    Trophy Points:
    0
    #13
    Submitting site map would do a great job
    Also if you plan like 1000/pages weekly then it would be easy (Submit those to google ,yahoo,msn )
    Put 1000 pages online.... Interlink each other and get some links for inner pages
    In 15 weeks you may get more traffci than you expect
     
    web-fanatic, Dec 6, 2008 IP
    axlarry likes this.
  14. tolgafiratoglu

    tolgafiratoglu Active Member

    Messages:
    249
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #14
    OK, I've just submitted my sitemaps in XML in developer account, it's pending. Let's see the results...
    I will inform you.

    Thanks :)
     
    tolgafiratoglu, Dec 6, 2008 IP
  15. tolgafiratoglu

    tolgafiratoglu Active Member

    Messages:
    249
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #15
    ALso, may I ask: What should I expect for a flickr-like photo gallery with this amount of content? I mean in terms of DAU? It's a very very heterogeneous photo gallery, including super award winning photos to normal item photos. I mean, one of the photos may have huge traffic and the other may be unvisited for months.

    I guess you'll relax on your timing expectations, because images are easier to handle for Google, it's not text content

    Thanks again
     
    tolgafiratoglu, Dec 6, 2008 IP
  16. greatme

    greatme Active Member

    Messages:
    789
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    80
    #16
    Try to build Good Backlinks but understand Search engines does not index so fast and not all pages indexed by them.
     
    greatme, Dec 6, 2008 IP
  17. tolgafiratoglu

    tolgafiratoglu Active Member

    Messages:
    249
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #17
    OK. URLs were submitted.

    I'll wait for the index results.

    Thanks for your help.
    Tolga
     
    tolgafiratoglu, Dec 6, 2008 IP
  18. tolgafiratoglu

    tolgafiratoglu Active Member

    Messages:
    249
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #18
    Only if you don't have a high PR :) - mine is PR:0

    You know DP, Facebook, or Digg are indexed very very frequently. I'm very serious that I see what I write in Digg in about 10-20 minutes. :)
     
    tolgafiratoglu, Dec 6, 2008 IP
  19. tarunkumar_model

    tarunkumar_model Well-Known Member

    Messages:
    455
    Likes Received:
    121
    Best Answers:
    0
    Trophy Points:
    113
    #19
    Its almost next to impossible to get all of your 15K pages indexed in Google within 3-4 days

    However, You can speed up the process of indexing by following the methods listed below:
    1. Submit SiteMap pages
    2. Make Pages Linking to 100s of your Inner Pages
    3. Put Links to those Pages on Your Home Page
    4. Social Bookmark all of those link pages
    5. DO Link Building to those pages and your home page

    Google will find these pages with links to other pages and then can crawl its way further deeper into the structure easily and quickly

    This method can help you reach better indexing %ages but it will still take time
     
    tarunkumar_model, Dec 6, 2008 IP
  20. tolgafiratoglu

    tolgafiratoglu Active Member

    Messages:
    249
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #20
    Well, they already crawled 4K of them in a day? I guess they do it faster recently.
     
    tolgafiratoglu, Dec 8, 2008 IP