1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

how can i start my own search engine?

Discussion in 'All Other Search Engines' started by i_am_dhaval, May 24, 2007.

  1. schlogo

    schlogo Well-Known Member

    Messages:
    1,615
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    140
    #41
    would you know and existing script?. Talking script, what is the one running on la dump? :D
    SEMrush
     
    schlogo, May 25, 2007 IP
    SEMrush
  2. projectshifter

    projectshifter Peon

    Messages:
    394
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #42
    As far as an existing search engine type script I can't think of any that I'm aware of, I do pretty much custom work vs. dealing with prebuilt things, but a simple search engine on a small list of things would be pretty simple. As far as the script on ladump, that's custom by me, but if you want to be technical it's more or less a pre-release of my Finale CMS that I'm going to be releasing later today. It's a prebuilt system, but it's flexible to the point you could do anything from a flash game site, to a collegehumor type site, or a youtube site with or without flash videos, but it's all the perks of custom because it "writes" the code for you depending on what options you choose so it keeps all the benefits of having it custom coded from someone who doesn't slack off and write shitty code :p
     
    projectshifter, May 26, 2007 IP
  3. davidelvar

    davidelvar Well-Known Member

    Messages:
    568
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    130
    #43
    davidelvar, May 26, 2007 IP
  4. schlogo

    schlogo Well-Known Member

    Messages:
    1,615
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    140
    #44
    thanks a lot, a link to your cms would be welcome by pm if you want. I can betatest it if you need :D
     
    schlogo, May 26, 2007 IP
  5. schlogo

    schlogo Well-Known Member

    Messages:
    1,615
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    140
    #45
    thanks for the url :)
     
    schlogo, May 26, 2007 IP
  6. Kalyse

    Kalyse Peon

    Messages:
    1,221
    Likes Received:
    24
    Best Answers:
    0
    Trophy Points:
    0
    #46
    Anyone else laugh at this post?
    Some questions are just plain stupid.
    If you have to ask 'how' for some things, then you have no idea how to do it.

    I have a question for people.
    How can I make a really successful business like Intel. Can someone help me?
     
    Kalyse, May 26, 2007 IP
  7. webmasterlabor.com

    webmasterlabor.com Peon

    Messages:
    2,889
    Likes Received:
    76
    Best Answers:
    0
    Trophy Points:
    0
    #47
    You need to be different..... How about a manual search engine? ie., REAL PEOPLE doing the searches.
     
    webmasterlabor.com, May 26, 2007 IP
  8. tbarr60

    tbarr60 Notable Member

    Messages:
    3,456
    Likes Received:
    125
    Best Answers:
    0
    Trophy Points:
    210
    #48
    First make some computer chips that are twice as fast as the one Intel made 1.5 years ago. Within 1.5 years double the speed of the chip. Repeat and sell them.

    That should do it.
     
    tbarr60, May 26, 2007 IP
  9. webmasterlabor.com

    webmasterlabor.com Peon

    Messages:
    2,889
    Likes Received:
    76
    Best Answers:
    0
    Trophy Points:
    0
    #49
    If you want to put in the TIME to do it, I say go for it. As long as you have the SKILLS you need, don't be afraid to start small and scale up.
     
    webmasterlabor.com, May 27, 2007 IP
  10. Absolutus

    Absolutus Peon

    Messages:
    233
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #50
    Go with a custom script is easy to build, I wrote mine in 2004 in a week with php and mysql, I wrote it to build topic directories, it was able to crawl 100.000 home page, probably 8000 daily.

    So for your requirements a php/mysql script could work pretty well.
     
    Absolutus, May 27, 2007 IP
  11. Mark0

    Mark0 Peon

    Messages:
    49
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #51
    I have made a crawler + search engine of sort (first link on the sign!), but it's for a small, small, small niche.
    Starting now like Google started (with great skills & ideas, but low resources), doing something web-wide, it's definitely not going to happens, IMHO.
    The web is a lot larger that the web-that-was.
     
    Mark0, May 29, 2007 IP
  12. livingearth

    livingearth Well-Known Member

    Messages:
    1,470
    Likes Received:
    83
    Best Answers:
    0
    Trophy Points:
    140
    #52
    Step 1.. Invade new area
    Step 2..Kill majority of native inhabitants, outlaw their religion and language
    Step 3..Raise TBarrian Flag and say it was all "manifest destiny"
     
    livingearth, May 29, 2007 IP
  13. schlogo

    schlogo Well-Known Member

    Messages:
    1,615
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    140
    #53
    Hello

    I finally tried sphider, it installed smoothly and is fairly easy to customise, I don't know if I will keep using it for ever but it s a good script to start with, and of course it is free

    http://cs.ioc.ee/~ando/sphider/
     
    schlogo, May 30, 2007 IP
  14. jmalinie

    jmalinie Well-Known Member

    Messages:
    460
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    148
    #54
    jmalinie, May 30, 2007 IP
  15. schlogo

    schlogo Well-Known Member

    Messages:
    1,615
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    140
    #55
    well, been there already...there are some limitations I do not want :rolleyes:
     
    schlogo, May 31, 2007 IP
  16. dopiitv

    dopiitv Well-Known Member

    Messages:
    1,556
    Likes Received:
    51
    Best Answers:
    0
    Trophy Points:
    165
    #56
    haha
    Well its alot of work and money, you need to offer something unique. Like blingo.com or that seach engine- begins with a M... maybe Mahulao sorry for not knowing, gets all ofits results from manual searching, or even chacha with the guides. Competeing directly with google isnt a good idea unless you have 11 billion dollars... Not trying to put you down- just trying to give you some ideas
    Thanks
    Mike
     
    dopiitv, May 31, 2007 IP
  17. schlogo

    schlogo Well-Known Member

    Messages:
    1,615
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    140
    #57
    depends what you want, it s a niche search engine, it s integrated in a website
     
    schlogo, Jun 1, 2007 IP
  18. jill_domains

    jill_domains Peon

    Messages:
    219
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    0
    #58
    Here is the cost breakdown given by one of the Nutch founders, four years back:
    (I have not edited it):

    A typical back-end machine is a single-processor box with 1 gigabyte
    of RAM, a RAID controller, and eight hard drives. The filesystem is
    mirrored (RAID level 1) and provides 1 terabyte of reliable storage.
    Such a machine can be assembled for a cost of about $3,000.

    One such back-end machine is required for every 100 million pages.
    Thus, to maintain an index of 1 billion pages requires 10 back-end
    machines, or about $30,000 in hardware.

    A typical front-end machine is a single-processor box with 4 gigabytes
    of RAM and a single hard drive. Such a machine can be assembled for
    about $1,000.

    The query-handling capacity of front-end machines varies, depending on
    how much each must search. For example, if each front-end machine is
    given 25 million pages to search, then each can perform about two
    searches per second. Thus, a 100-million-page index could be searched
    with four front-end machines ($4,000) while a 1-billion-page index
    requires 40 front-end machines ($40,000), but such configurations
    could still handle only two searches per second. In this case, access
    to a disk-resident index is the primary bottleneck.

    Query handling is more cost effective when primary index structures
    fit within RAM. In particular, if each front-end machine is required
    to handle only 2 million pages, then each can handle perhaps 50
    searches per second. In this configuration a 100-million-page index
    would require 50 front-end machines ($50,000) and a 1-billion-page
    index would require 500 machines ($500,000). This is half the cost per
    query of the first case. Here the bottleneck is primarily the CPU.
    Further search software optimizations can make this configuration even
    more cost effective.

    Note that as traffic increases, front-end hardware quickly becomes the
    dominant hardware cost.

    Thus far we have discussed only the raw hardware costs. In addition,
    there are hosting costs. These are primarily electricity (as consumed
    both directly by the hardware and by the air conditioning required to
    cool the hardware), bandwidth, and others (racks, network equipment,
    facility rental, etc.). Electricity dominates these costs, and
    together, these costs easily dominate raw hardware costs. For example,
    you might amortize the cost of hardware over three years, so that
    $100,000 of hardware is less than $3,000 per month; but power, space,
    and bandwidth for 100 machines can easily cost more than that. Since
    hosting costs are even more variable than hardware prices, let's just
    assume that hosting costs are approximately the same as three-year
    amortized hardware costs. Thus, a complete system might cost anywhere
    between $800 per month for two-search-per-second performance over 100
    million pages, to $30,000 per month for 50-page-per-second performance
    over 1 billion pages.
    =====================
     
    jill_domains, Jun 2, 2007 IP
    schlogo likes this.
  19. bdmweb

    bdmweb Banned

    Messages:
    100
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #59
    I've always wanted to set up a search engine too, if you need someone else on board then let me know.
     
    bdmweb, Jun 2, 2007 IP
  20. bdmweb

    bdmweb Banned

    Messages:
    100
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #60
    What's the URL of your search engine?
     
    bdmweb, Jun 2, 2007 IP