handling php with large database

Discussion in 'PHP' started by ascentlogic, Aug 26, 2010.

  1. #1
    Hi, i am php programmer , i need help from php expert to create php apllication for large database .
    I have database table called "profiles" which contains billions(1.5 to 2 billion) of profile of the business companies. This table has 10 fields and there is one field named as "name" which is name of company , i made this column full-text index for full-text search .

    Now , i have to use this table for profile searching (using full-text search), profiles within particular cities , profiles within particular categories etc. This table contains billions of records so it will take lots of time for searching and fetching the reocrd(s) from this table.
    Can anybody help me that how can i manage this large table to improve the performance and fast searching with php ? Is there any other technique (algorithm) to manage large database (like facebook,twiiter,orkut)

    following is the table structure

    CREATE TABLE IF NOT EXISTS `profiles` (
    `id` int(11) NOT NULL AUTO_INCREMENT,
    `cityid` int(11) NOT NULL,
    `cid` int(11) NOT NULL,
    `name` varchar(200) NOT NULL,
    `phone` varchar(20) NOT NULL,
    `fax` varchar(20) NOT NULL,
    `address` varchar(1000) NOT NULL,
    `city` varchar(1000) NOT NULL,
    `state` varchar(100) NOT NULL,
    `zip` varchar(20) NOT NULL,
    `county` varchar(100) NOT NULL,
    `contact` varchar(100) NOT NULL,
    `gender` varchar(10) NOT NULL,
    `jobtitle` varchar(1000) NOT NULL,
    `website` varchar(100) NOT NULL,
    `email` varchar(100) NOT NULL,
    `emplayees` int(11) NOT NULL,
    `sales` int(100) NOT NULL,
    `type` varchar(1000) NOT NULL,
    `maincat` varchar(1000) NOT NULL,
    `url` varchar(500) NOT NULL,
    PRIMARY KEY (`id`),
    KEY `city` (`city`),
    KEY `cityid` (`cityid`),
    KEY `cid` (`cid`),
    FULLTEXT KEY `name` (`name`)
    )

    Queries are :
    1) select p.id,p.name,p.address,p.cityid,p.url,c.city,s.state,match(p.name) against('keywords') as score from profiles p,cities c,states s where 1=1 and p.cityid=c.id and c.stateid=s.stateid and match(p.name) against('keywords' in boolean mode) order by score desc

    2) select p.id,p.name,p.address,p.cityid,p.url,c.city,s.state from profiles p,cities c,states s where 1=1 and p.cityid=c.id and c.stateid=s.stateid and cid=2 order by p.name asc

    3) select p.id,p.name,p.address,p.cityid,p.url,c.city,s.state from profiles p,cities c,states s where 1=1 and p.cityid=c.id and c.stateid=s.stateid and s.stateid=33 and cityid=7810 order by p.name asc
     
    Last edited: Aug 26, 2010
    ascentlogic, Aug 26, 2010 IP
  2. Rainulf

    Rainulf Active Member

    Messages:
    373
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    85
    #2
    PHP isn't really best for handling huge database like that because business information such as that is usually done in IBM's iSeries (if you are consider security as well). Websites such as Twitter or FaceBook don't use PHP, I believe Twitter uses Ruby and I heard Google uses Python, though I could be wrong.

    You might also wanna check the performances between SQL engines: http://www.mysqlperformanceblog.com/2007/01/08/innodb-vs-myisam-vs-falcon-benchmarks-part-1/ - that might help
     
    Rainulf, Aug 26, 2010 IP
  3. lowridertj

    lowridertj Well-Known Member

    Messages:
    2,882
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    195
    #3
    Python is very powerful..
     
    lowridertj, Aug 26, 2010 IP
  4. canishk

    canishk Peon

    Messages:
    46
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    FYI facebook is a PHP app
     
    canishk, Aug 27, 2010 IP
  5. TheWiseGuy

    TheWiseGuy Member

    Messages:
    113
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    28
    #5
    exactly, and MySQL.

    Make sure to create indexes on the columns used in query conditions.

    See which queries are slow and optimize from there.
     
    TheWiseGuy, Aug 27, 2010 IP
  6. so_sweet

    so_sweet Member

    Messages:
    234
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    28
    #6
    how about http://www.facebook.com/editprofile.php ?
     
    so_sweet, Aug 27, 2010 IP
  7. lowridertj

    lowridertj Well-Known Member

    Messages:
    2,882
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    195
    #7
    not entirely.

    its built using many coding types however the mix of them all is what makes up the facebook ui
     
    lowridertj, Aug 27, 2010 IP