ETL : from delimited files to mysql database

Discussion in 'MySQL' started by truster, Jul 10, 2008.

  1. #1
    Hi all,

    Here is my problem: I have several delimited files and I want to transfer their content to a MySQL database.

    Before asking on this forum, I've been browsing the net and I heard about ETL tools that are open source and easy to use for non technical users. Can these products really be used by people who don't have much technical background and which one do you think is the most user-friendly ?

    Thanks.
     
    truster, Jul 10, 2008 IP
  2. T.Guru

    T.Guru Peon

    Messages:
    78
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Hello, truster, indeed, ETL tools are able to perform the transfer you're describing.
    And Talend, my employer, produces such a tool.

    Since you describe yourself as a non technical user, I won't waste your time with technical considerations. Simply put,Talend Open Studio is an open-source ETL tool with a user-friendly GUI which is easy to learn for a non-technical user. But what really distinguishes Talend Open Studio from other tools, when it comes to business users, is the tMap component which allows the user to get a graphical and functional view of integration processes.

    Now back to your question, do you only have CSV files or also other files (excel sheets for instance) ?
     
    T.Guru, Jul 11, 2008 IP
  3. truster

    truster Peon

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Thanks for your answer. I'm going to have a look at Talend Open Studio, and try to give a feedback asap.

    And yes the files I want to load in the MySQL database are all csv files.
     
    truster, Jul 15, 2008 IP
  4. T.Guru

    T.Guru Peon

    Messages:
    78
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Well ok then.

    By the way, if you have only csv files you may also import them directly in your mySQL database with :

    LOAD DATA INFILE 'path/file.txt' INTO TABLE your_table;
     
    T.Guru, Jul 16, 2008 IP
  5. truster

    truster Peon

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Hi there,

    I've been trying Talend Open Studio and it can do much more things than I thought...

    What's more, with such a nice GUI (and the "job designer" part) it is really easy to understand the transfer/transformation/loading/synchronization/... you're implementing.

    Really a nice program, but it may be difficult to use on an old pc.
     
    truster, Jul 20, 2008 IP
  6. T.Guru

    T.Guru Peon

    Messages:
    78
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Did you manage to do what you wanted ?

    I am glad to hear that you were able to use the program smoothly.

    Concerning the "old pc" matter, you can use Talend Open Studio on old computers but the jobs will take more time to execute themselves, that's all.
     
    T.Guru, Jul 21, 2008 IP
  7. truster

    truster Peon

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Yes I managed to do it !

    The program is really easy to use for business users, but some friend of mine (more of a technical user than me) told me that there was a "java debugger". Is that right ?
     
    truster, Jul 21, 2008 IP
  8. T.Guru

    T.Guru Peon

    Messages:
    78
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Indeed Talend Open Studio does have a "java debugger".

    But such features as "java debugger" or "code injection" are more intended for technical users (that's why your friend noticed it straightaway).

    Simply put, TOS is to be used by business users (for it has a user-friendly GUI, easy to master) but also by technical users (for it has features like a "java debugger")

    If you need extra information about the "java debugger" you can have a look at the User Guide.

    And don't hesitate if you have other questions ;)
     
    T.Guru, Jul 22, 2008 IP
    truster likes this.
  9. truster

    truster Peon

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Thanks for everything, I won't hesitate if I have other projects needing the use of Talend Open Studio.
     
    truster, Jul 28, 2008 IP
  10. T.Guru

    T.Guru Peon

    Messages:
    78
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Yeah don't hesitate, since it has many features and encompasses the whole ETL process you might need to use TOS again in future projects ;-)
     
    T.Guru, Aug 1, 2008 IP
  11. T.Guru

    T.Guru Peon

    Messages:
    78
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #11
    For more information and to get answers about Talend, I would suggest you look at Talendforge. The site, well documented, will be able to help you : http://www.talendforge.org/
     
    T.Guru, Jan 19, 2009 IP