200 000 queries - do not execute all of them

Discussion in 'Databases' started by heatdesign, Oct 7, 2009.

  1. #1
    I have this problem with sending 200 000 times the query "SELECT * FROM emails WHERE title = '".$el."'" to check if the email already exist in the DB. It looks like this : foreach ($emails as $el) {
    $query = "SELECT * FROM emails WHERE title = '".$el."'";
    here goes the check...
    }
    but if i put a counter in "foreach" without the $query, it goes through all elemenets, unfortunately if I send a query every time it can't reach the end of the cycle... with 100 000 emails there is no problem... what am i doing wrong?
     
    heatdesign, Oct 7, 2009 IP
  2. jestep

    jestep Prominent Member

    Messages:
    3,659
    Likes Received:
    215
    Best Answers:
    19
    Trophy Points:
    330
    #2
    Are you doing this before adding them to your database?

    You could do a conditional insert if that is the case and the emails in there wont get added. You could also create a unique index on the email column which would prevent duplicates.
     
    jestep, Oct 7, 2009 IP
  3. premiumscripts

    premiumscripts Peon

    Messages:
    1,062
    Likes Received:
    48
    Best Answers:
    0
    Trophy Points:
    0
    #3
    What error do you get? Is it simply a case of PHP exceeding the time limit?

    Use set_time_limit(0); to prevent that.

    Otherwise, you may want to change your query so that you process the emails in chunks of 100 and use WHERE title IN (' . implode(',', $email_chunk) . ')';

    (You would have to already add " and mysql_real_escape_string before you do that though.)
     
    premiumscripts, Oct 8, 2009 IP
  4. heatdesign

    heatdesign Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    PHP exceeding the time limit... that was the problem! and set_time_limit(0); made it work :) thanks a lot!
     
    heatdesign, Oct 8, 2009 IP