1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How do i COUNT how many pages a website has?

Discussion in 'Site & Server Administration' started by gmantakis, Oct 23, 2006.

  1. #1
    Hi to all,

    I want to count for a specific website how many pages with content it has. The website claims to have 1000 pages of content but i want to confirm that. Is there any software or online tool for this? For example i want to enter www.exampledomain.com and then get in return a number like 323 which would mean that this domain has 323 pages AND 200 other outgoing links to other domains.

    If there isnt a software or online tool can i get these results with some specific search pattern from Google?

    Thanks in advance

    Georg
    gmantakis, Oct 23, 2006 IP
  2. tunedtv

    tunedtv Peon

    Messages:
    219
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #2
    that's not too easy to do. If they're html documents you might be able to use find and wc on the unix command prompt.
    tunedtv, Oct 24, 2006 IP
  3. Mystique

    Mystique Well-Known Member

    Messages:
    2,509
    Likes Received:
    80
    Best Answers:
    2
    Trophy Points:
    145
    #3
    If it is you site you can count them on you PC via software.

    Your hard copy can be counted by pages and links, but hard enough if you want to figure other's site.
    Mystique, Oct 24, 2006 IP
  4. sweb

    sweb Peon

    Messages:
    375
    Likes Received:
    25
    Best Answers:
    0
    Trophy Points:
    0
    #4
    try XENU - the program will check for links and you'll get idea about nearly all pages on the site. Another tip - if the site uses some popular content mangment system and is not seo optimised - look for strings like - id=4,id=290
    and by playing guess the number game you may find all pages.

    http://home.snafu.de/tilman/xenulink.html
    sweb, Oct 24, 2006 IP
  5. Pat Gael

    Pat Gael Banned

    Messages:
    1,332
    Likes Received:
    68
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Xenu is great indeed, one of the first I tried when I just started with the Internet almost ten years ago.

    I didn't know its development was continued until today.
    Pat Gael, Oct 24, 2006 IP
  6. gmantakis

    gmantakis Member

    Messages:
    54
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    43
    #6
    I tried XENU but what i noticed is that XENU counts a page as many times as it is linked internally? For example if we have the domain http://www(.)competitordomain(.)com/ and the site has 500 pages. If all of the 500 pages have a LINK back to the root page http://www(.)competitordomain(.)com/index.html then i think XENU counts that 500 links as separate pages? I am trying to check a domain and it outputs that it has 4000 pages... and it has clearly not so many pages. So isnt there any other easy way to count how much content a domain has? I just want to count the different HTML pages.. i dont want to link check a domain.

    Thanks

    George
    gmantakis, Oct 29, 2006 IP
  7. ilovecoupons

    ilovecoupons Peon

    Messages:
    3
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Hi,

    You can try http://gsitecrawler.com/

    I have used it for http://www.ilovecoupons.com.au and http://www.loveacoupon.com and it was very accurate.

    Dom
    ilovecoupons, Dec 4, 2010 IP
  8. atxsurf

    atxsurf Peon

    Messages:
    2,399
    Likes Received:
    21
    Best Answers:
    1
    Trophy Points:
    0
    #8
    you could try to crawl it using recursive wget, than count the number of files collected
    atxsurf, Dec 5, 2010 IP
  9. VINEETYADAV

    VINEETYADAV Peon

    Messages:
    28
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    best option as i try

    goto a site that creates sitemap for google.
    crawl site and you will get number of poages in the site
    VINEETYADAV, Dec 9, 2010 IP
  10. ilovecoupons

    ilovecoupons Peon

    Messages:
    3
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Which sitemap creation software would you recommend to use for that?
    ilovecoupons, Dec 9, 2010 IP
  11. VINEETYADAV

    VINEETYADAV Peon

    Messages:
    28
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    use online sitemap creator.
    Search on google for "Free Google SItemap Creator" and you will get links.
    use anyone. they will crawl to all pages and in finalr report will tell you details
    VINEETYADAV, Dec 9, 2010 IP
  12. ilovecoupons

    ilovecoupons Peon

    Messages:
    3
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Yes, I have previously done this for www.ilovecoupons.com.au and the results were not great as they were a lot of duplicate urls for votes up and down, print, e-mail. It's why unless you have some specific sitemap software that cleans all of the rubbish urls I think Gsitecrawler is a better option.
    ilovecoupons, Dec 9, 2010 IP
  13. xzgz0305

    xzgz0305 Peon

    Messages:
    86
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #13
    You can link to see it?
    xzgz0305, Dec 10, 2010 IP
  14. buckplus

    buckplus Greenhorn

    Messages:
    13
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    21
    #14
    The easiest way to do that is by creating a sitemap with "Google sitemap Generator" at the end it will give your details on the right how many pages were scanned. hope this helps.
    buckplus, Dec 11, 2012 IP
  15. Vincentas

    Vincentas Greenhorn

    Messages:
    67
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    18
    #15
    Usign a sitemap is the best option imho.
    Vincentas, Jul 17, 2013 IP
  16. gkd_uk

    gkd_uk Well-Known Member

    Messages:
    902
    Likes Received:
    64
    Best Answers:
    1
    Trophy Points:
    115
    #16
    This thread is from 2006
    gkd_uk, Jul 17, 2013 IP
  17. Vincentas

    Vincentas Greenhorn

    Messages:
    67
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    18
    #17
    Wow, how did it get bumped in the first place? Sorry for that.
    Vincentas, Jul 20, 2013 IP