Incomplete database population

Discussion in 'Databases' started by SitPoMk, Jan 16, 2008.

  1. #1
    hey there

    I have a script which runs daily to update my DB with data(online games)
    It works like this, opens an XML feed from my provider, parses it, and runs individual mysql_querys to insert rows into my games table

    The problem is that only around half of these games get put into the database and these missing items are not in a particular order. I actually cannot find any kind of pattern or similarity between the ones that do not end up in the DB as opposed to the ones that do.

    I hope this is enough information, if not please inform me.

    Has anyone had something similar?
     
    SitPoMk, Jan 16, 2008 IP
  2. Marty_

    Marty_ Banned

    Messages:
    1,031
    Likes Received:
    44
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Perhaps its your different database fields length, like maybe games with a long description are blocked or a long title. It could be anything, maybe the game ids are clashing with others you have listed already, who knows.

    Could you show us the page where the data comes from and of course your page so we can compare the differences ?
     
    Marty_, Jan 17, 2008 IP
  3. mlkshake

    mlkshake Peon

    Messages:
    73
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yes, we need the to see what your describing. Its probably some data that your picking up thats messing with your queries. Try using escape characters in your query string.
     
    mlkshake, Jan 17, 2008 IP
  4. awatson

    awatson Active Member

    Messages:
    112
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    53
    #4
    Can you make your script log the queries it's using? Then you can find the queries for the items that didn't get inserted and see if anything is wrong with them.
     
    awatson, Jan 17, 2008 IP
  5. bluegrass special

    bluegrass special Peon

    Messages:
    790
    Likes Received:
    50
    Best Answers:
    0
    Trophy Points:
    0
    #5
    It would also be helpful to see one of the xml files and show what records did not get entered. That with the table structure (field type, size, etc...) would be more helpful in determining what might cause the error(s).
     
    bluegrass special, Jan 17, 2008 IP
  6. SitPoMk

    SitPoMk Peon

    Messages:
    457
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #6
    sure! sorry guys i was on vacation
    my page is http://onlinegalore.com
    here is the XML file it is using: XML

    for example you may see on my front page that my database only contains 2 mahjong games
    The xml file however contains more than that

    once again sorry for late late response

    looking forward to yours ;)
     
    SitPoMk, Jan 24, 2008 IP
  7. sharqi

    sharqi Guest

    Messages:
    105
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #7
    logging - log your results and all will become clear i suspect.
     
    sharqi, Jan 25, 2008 IP