crawl script

Discussion in 'Scripts' started by JEET, Mar 15, 2006.

  1. #1
    Hi,
    Anyone here knows a script that can be installed on a site's server and when run can crawl "another" domain and list the URL's of all files and subdirectories?
    It's really important and urgent . Kindly help.

    Thanks
    jeet
     
    JEET, Mar 15, 2006 IP
  2. temp_12000

    temp_12000 Peon

    Messages:
    316
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #2

    it is NOT easy to write a Good crawler. maybe you can try rentacoder.com


    Temp
     
    temp_12000, Mar 15, 2006 IP
  3. Colbyt

    Colbyt Notable Member

    Messages:
    3,224
    Likes Received:
    185
    Best Answers:
    0
    Trophy Points:
    210
    #3
    The fluid dynamics search Engine script will do just that very thing.

    If you can't find them from that info PM me and I will give you a link as I fomerly used it on my site and the files are still there.

    Colbyt
     
    Colbyt, Mar 15, 2006 IP
  4. JEET

    JEET Notable Member

    Messages:
    3,832
    Likes Received:
    502
    Best Answers:
    19
    Trophy Points:
    265
    #4
    Hi,
    I found the fluid dynamics search Engine but this is a search utility. What I am looking for is a script that will "list" all the files and folders for the domain entered.
    ( It need not be installed on the same domain)
    Like I install it on my domain and then enter a different domain in a box and click start. It then finds all the file names and folder names in that domain and puts them all in a textbox or simply prints it to a webpage.
    Can you suggest any?
    Thanks
    jeet
     
    JEET, Mar 15, 2006 IP
  5. sarahk

    sarahk iTamer Staff

    Messages:
    28,821
    Likes Received:
    4,539
    Best Answers:
    123
    Trophy Points:
    665
    #5
    No script will be able to list all the files and folders.

    You can easily get a script to list all the pages but with mod_rewrite and dynamic coding that is very different to "files and folders".

    Just download joomla and compare the files in the download to the pages on a joomla site. There's no relationship between the two.
     
    sarahk, Mar 16, 2006 IP
  6. Colbyt

    Colbyt Notable Member

    Messages:
    3,224
    Likes Received:
    185
    Best Answers:
    0
    Trophy Points:
    210
    #6
    I guess I did not read what you wrote carefully enough.
     
    Colbyt, Mar 16, 2006 IP