1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Tool to crawl my site and put my URLs in a database

Discussion in 'PHP' started by netaddict, Nov 6, 2005.

  1. #1
    Hope you people are doing well :)

    Is there any script or tool that can crawl my entire site and put all of my URLs with their respective "Titles" in a MySQL database?

    I will be really really thankful, if someone can help me out on this. I can even give a free PR5 link for six months, if someone helps me achieve that.

    Thanks in advance.
     
    netaddict, Nov 6, 2005 IP
  2. piniyini

    piniyini Well-Known Member

    Messages:
    514
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    170
    #2
    I've got this script on my comp (somewhere) will dig it out when I get my hands on it!

    Oops! Misread your post, thought you wanted the urls or title in a db when a crawler/bot visits your site. Ignore the above, sorry.
     
    piniyini, Nov 6, 2005 IP
  3. netaddict

    netaddict Peon

    Messages:
    640
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Waiting desperately for it :p
     
    netaddict, Nov 6, 2005 IP
  4. piniyini

    piniyini Well-Known Member

    Messages:
    514
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    170
    #4
    Aw making me feel bad now :eek:

    How many pages do you have on your website which you need the titles of?
     
    piniyini, Nov 6, 2005 IP
  5. netaddict

    netaddict Peon

    Messages:
    640
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Ah! don't :eek:

    Not much. Just around 400-500.
     
    netaddict, Nov 6, 2005 IP
  6. piniyini

    piniyini Well-Known Member

    Messages:
    514
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    170
    #6
    Is it ok if I chat to you on msn? It woudl be quicker this way ...
     
    piniyini, Nov 6, 2005 IP
  7. netaddict

    netaddict Peon

    Messages:
    640
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I am sorry but piniyini was not able to help me, though he is a nice person.
    Anyone else?
     
    netaddict, Nov 6, 2005 IP
  8. WhatiFind

    WhatiFind offline

    Messages:
    1,789
    Likes Received:
    257
    Best Answers:
    0
    Trophy Points:
    180
    #8
    WhatiFind, Nov 6, 2005 IP
  9. hnn

    hnn Peon

    Messages:
    91
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Did you worked it out?
     
    hnn, Nov 7, 2005 IP
  10. netaddict

    netaddict Peon

    Messages:
    640
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    0
    #10
    I was not able to get that isearch to work out (thanks though), but, I searched and searched and finally come across "Sphider" - I found it cool and was able to do what I was looking for :) works like a charm :p

    Heres the link - http://www.cs.ioc.ee/~ando/sphider/
     
    netaddict, Nov 7, 2005 IP
  11. deepnuke

    deepnuke Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    Newbie on PHP
     
    deepnuke, Nov 10, 2005 IP