1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How to index a huge site?

Discussion in 'Bing' started by mahmood, Jun 12, 2006.

  1. #1
    I wonder if anybody here has a good experience with MSN and huge sites.

    As far as I have seen MSN indexes pages with a direct link on the homepage or second level distance from it, but when it comes to several level, MSN just leave it.

    When we have a huge site - say 300K pages - we cannot simply put a link to every page at first level or second level.

    Do you have any suggestions to help me get a huge site indexed at MSN.
     
    mahmood, Jun 12, 2006 IP
    hsaleem likes this.
  2. ghosthost

    ghosthost Peon

    Messages:
    524
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #2
    how about sitemap and you can put the link in 2 level deep..
     
    ghosthost, Jun 12, 2006 IP
  3. mahmood

    mahmood Guest

    Messages:
    1,228
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I think 2 levels wouldn't work for 1000K pages. If I put 1000 links on each page then I need 1 sitemap containing 1000 links to pages each containing 1000 links.
    Wouldn't you say 1000 links on each page is not a good idea?
     
    mahmood, Jun 12, 2006 IP
  4. vic_msn

    vic_msn Well-Known Member

    Messages:
    2,233
    Likes Received:
    125
    Best Answers:
    0
    Trophy Points:
    155
    #4
    MSN indexes pretty fast and ranks you higher initially it also indexes dynamic links better than google or yahoo. the best possible way is to make a sitemap.xml and urllist.txt to index all your links also try searching in other adsense search boxes.
     
    vic_msn, Jun 12, 2006 IP
  5. arnoldcr

    arnoldcr Peon

    Messages:
    125
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    for me its been fast with sitemaps
     
    arnoldcr, Jun 13, 2006 IP
  6. mahmood

    mahmood Guest

    Messages:
    1,228
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Hummm....
    I try to find some way to sitemapise my site.
     
    mahmood, Jun 13, 2006 IP
  7. Savweb

    Savweb Guest

    Messages:
    398
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #7
    You need tons of backlinks to get indexed well
     
    Savweb, Jun 14, 2006 IP
  8. mahmood

    mahmood Guest

    Messages:
    1,228
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Backlinks to homepage or each page?
     
    mahmood, Jun 14, 2006 IP
  9. BFTUK

    BFTUK Banned

    Messages:
    685
    Likes Received:
    19
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Interesting thread! I have a site with 11k + pages yet the SEs seem to have only approx 150 indexed each.
     
    BFTUK, Jun 14, 2006 IP
  10. Savweb

    Savweb Guest

    Messages:
    398
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Backlink to homepage is always better in global point of view.
    And and in this particular case it is better to get more backlinks exactly to your homepage.
     
    Savweb, Jun 14, 2006 IP
  11. mahmood

    mahmood Guest

    Messages:
    1,228
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #11
    Would you share some of these advantages?
     
    mahmood, Jun 14, 2006 IP
  12. BFTUK

    BFTUK Banned

    Messages:
    685
    Likes Received:
    19
    Best Answers:
    0
    Trophy Points:
    0
    #12
    So it looks like we need to get the search engines to dig deeper into our sites.

    How do we do that?

    Alot of good quality links pointing to our homepage?
     
    BFTUK, Jun 14, 2006 IP
  13. seoquiz

    seoquiz Peon

    Messages:
    33
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #13
    That way of linking will get you ranked highly for your homepage but it may take a bit for PR to spread to your other pages and get them ranked highly.

    Another method of link building, deep content link building has you link building for internal pages so they can rank highly. This is impossible for every page of your large site but you could still do it to some internal pages you want to rank highly for.
     
    seoquiz, Jun 14, 2006 IP
  14. ForgottenCreature

    ForgottenCreature Notable Member

    Messages:
    7,458
    Likes Received:
    170
    Best Answers:
    0
    Trophy Points:
    260
    #14
    Indeed, just look at DP - it has over 4 million + links in.
     
    ForgottenCreature, Jun 14, 2006 IP
  15. Dazed_and_confused

    Dazed_and_confused Well-Known Member

    Messages:
    2,071
    Likes Received:
    60
    Best Answers:
    0
    Trophy Points:
    120
    #15
    do you mean html sitemaps or does msn have sitemap tool ?
     
    Dazed_and_confused, Jun 14, 2006 IP
  16. Linknz

    Linknz Peon

    Messages:
    20
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #16
    It may be worth you getting some links that point to some of the pages that are deeper in your website.That way many spiders will find the link into your site and start indexing from the inside outwards will get more content indexed that is deeper.
    I could also be that they were just doing a light crawl and not going deep.
     
    Linknz, Jun 14, 2006 IP
  17. andy_boyd

    andy_boyd Active Member

    Messages:
    330
    Likes Received:
    33
    Best Answers:
    0
    Trophy Points:
    83
    #17
    Structure

    For MSN, try to keep the structure of your site as flat as possible. MSNbot seems to skim a site and seems very reluctant to venture any deeper than the second directory. Avoid this from the start by optimizing your site's structure to be as lean, mean and flat as possible.


    ... rather than ...


    Optimizing Navigation
    Use crumbs to provide a navigation funnel / tree where users and bots can move vertically (or horizontally as your content will all be in the same root directory).

    • Link back to the homepage from the main category
    • Link back to the homepage and main category the subcategory
    • Link back to the homepage, main category and subcategory from the article

    Inbound Links
    Point plenty of links to your homepage to keep MSNbot visiting. Install a randomizer on the homepage that will generate something unique (a new article / list of new content or products etc.) so the page always has fresh content. Link to both your main categories and subcategories from the homepage. Moreover, link to prominent articles that will encourage the bot to dig deeper.

    Then point good solid inbound links to your category, subcategory and article pages. It is especially important to take advantage these deep links (with on-topic anchor text) as they will provide a deeper point of entry to your site for the bot. It is typically easier to get 'deeper' content indexed when the bot lands right in the thick of it.

    A few other things off the top of my head that might help ...

    • Always provide a navigation crumb.
    • Replicate and image-based navigation with a textual equivalent.
    • Keep it fresh, if you use a blogging platform then schedule posts for release every day.
    • Use a 301 redirect to establish the site's canonical domain.
    • When using WordPress with clean URLs, add the following to your robots.txt to avoid confusing the bot ...

      User-agent: *
      Disallow: /?p

    I'm sure there's a load more, and every DP member will know this anyway. But if it helps someone then that's cool! :)
     
    andy_boyd, Jun 14, 2006 IP
  18. IamNed

    IamNed Peon

    Messages:
    2,707
    Likes Received:
    276
    Best Answers:
    0
    Trophy Points:
    0
    #18

    Um ..no..

    here is HOW you index LOTS of pages whihc actually works and is much simpler than the long rambling explanation above.


    Get an inbound PR7+ link to the sitemap. Wait a few weeks for the entire site to index. A link will cost 50-150/month but it is the fastest way and very easy to do.
     
    IamNed, Jun 14, 2006 IP
  19. windy

    windy Active Member

    Messages:
    1,093
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    78
    #19
    use sitemap tools
     
    windy, Jun 14, 2006 IP
  20. av1

    av1 Active Member

    Messages:
    168
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    60
    #20
    is there a place in msn to submit a sitemap like there is in google?
     
    av1, Jun 14, 2006 IP