Restoring a massive database?

Discussion in 'MySQL' started by cpvr, Jan 27, 2006.

  1. #1
    For my site Sweet Lyrics, I've only managed to restore 40 MB of a 280 MB database, how would I go about restoring the rest of it?

    I've tried uploading the .sql via the upload sql thing in cpanel, however; that didn't work, so I tried uploading a .txt via phpmyadmin - and that didn't work.
     
    cpvr, Jan 27, 2006 IP
  2. The Big Deal

    The Big Deal Peon

    Messages:
    212
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Phpmyadmin is no good for big files, you need to do this:

    -FTP the file up to Host
    -SSH into your Host

    If the database is NOT compressed:
    mysql -u username -p password DATABASE-NAME < database.sql

    Even better, if you have it compressed:
    gunzip < database.gz | mysql -u username -p password DATABASE-NAME
     
    The Big Deal, Jan 27, 2006 IP
  3. cpvr

    cpvr Guest

    Messages:
    1,030
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Thanks, I'll try it - will post the update.
     
    cpvr, Jan 27, 2006 IP
  4. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #4
    Alternatively, especially if you can't use SSH, try BigDump: The Staggered MySQL Dump Importer

     
    minstrel, Jan 28, 2006 IP
  5. cpvr

    cpvr Guest

    Messages:
    1,030
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Thanks for the help guys, very appreciated! :)
     
    cpvr, Jan 28, 2006 IP
  6. wwm

    wwm Peon

    Messages:
    308
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #6
    im not sure of the structure of your database

    but what i use regulary to transfer and backup large database (700-800mb) is Naviat 7

    its very usefull for mysql
     
    wwm, Jan 28, 2006 IP
  7. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #7
    Do you have a link for that? It might have saved me about three days of grief with one particularly troublesome MySQL database... :eek:
     
    minstrel, Jan 28, 2006 IP
  8. cpvr

    cpvr Guest

    Messages:
    1,030
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Heh, I've been working on this MySQL database for about a day now. =/
     
    cpvr, Jan 28, 2006 IP
  9. dvduval

    dvduval Notable Member

    Messages:
    3,372
    Likes Received:
    356
    Best Answers:
    1
    Trophy Points:
    260
    #9
    Yes, as Minstrel said, Big Dump is a great program, and saves lot of time.
    As a matter of fact, I am restoring a 1.71 GB database right now.
     
    dvduval, Jan 28, 2006 IP
  10. wwm

    wwm Peon

    Messages:
    308
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #10
    sorry i mistyped its

    NAVICAT 7

    http://www.navicat.com/

    (i cant put a live link :eek: )
    anyways i use navicat daily from running backups to designing tables to exporting/importing data to/from any data format under the sun

    also i love the fact that u can remotely connect thru ssh and edit remote database as it was here (ull need to set mysql on the server to receive connections from ur home ip address)

    the coolest part is being able to connect to several of my servers at the same time, and move and synchronise data between them, without copying the data to my home pc first

    (if u find it usefull send some green points my way ;) )
     
    wwm, Jan 29, 2006 IP
  11. Skribblez

    Skribblez Notable Member

    Messages:
    5,939
    Likes Received:
    208
    Best Answers:
    0
    Trophy Points:
    280
    #11
    Sorry to bump this. But I have a question. The database.sql file would have to be uploaded to the sites root dir, right? (The root dir is the directory before public_html)

    If not, where would I have to upload the database.sql file?
     
    Skribblez, Feb 11, 2006 IP
  12. The Big Deal

    The Big Deal Peon

    Messages:
    212
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Sure, you can put it in the root but it should not matter where it is as long as you run the command from the location of the file so it can be found or use the path to the file in the command.
     
    The Big Deal, Feb 11, 2006 IP
  13. arnek

    arnek Active Member

    Messages:
    134
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #13
    I prefer Navicat, can't get anything better.

    MySQL Query Browser is also nice, but Navicat has very nice features for different kinds of dumps etc.

    I vote Navicat
     
    arnek, Feb 13, 2006 IP
  14. RazorWing

    RazorWing Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    I have a problem with my SQL Database, i tryed uploading it with phpbbadmin, that didnt work, i tryed uploading it with phpmyadmin, that didnt work, then i tryed uploading it with bigdump, it half worked, but generated an hour. Is there any way to repair the database or bypass the error which i am getting?

    BigDump: Staggered MySQL Dump Importer ver. 0.21b
    Processing file: backup.sql

    Starting at the line: 1

    Error at the line 30397: INSERT INTO phpbb_search_wordlist (word_text, word_id, word_common) VALUES('Ã¥', '3189', '0');

    Query: INSERT INTO phpbb_search_wordlist (word_text, word_id, word_common) VALUES('Ã¥', '3189', '0');

    MySQL: Duplicate entry 'Ã¥' for key 1

    Stopped on error
     
    RazorWing, Apr 11, 2006 IP
  15. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #15
    You've got an error in the file that bigdump is processing.

    1. fix the error in your MySQL file: what is 'Ã¥' supposed to be? or you can just drop the search wordlist table for now and rebuild it later (that's just one of two or three tables that phpBB uses to assemble keywords and such for forum searches - there is a utility to let you rebuild the search and it starts by dropping the old tables anyway)

    2. empty your new database of data to avoid another error

    3. re-run bigdump
     
    minstrel, Apr 11, 2006 IP
  16. RazorWing

    RazorWing Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #16
    What utility do i use to rebuild the tables?

    I am kinda a noob at things like this, i have recently changed web hosting companys so i am having trouble moving the SQL database.

    Would you be able to guide me through the steps i have to take in order to fix the database problem i am having?
     
    RazorWing, Apr 12, 2006 IP
  17. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #17
    BigDump rebuilds the tables once you fix the errors in the SQL statements.

    Do you mean to rebuild the search tables? If so:

    http://www.phpbb.com/phpBB/viewtopic.php?t=329629

     
    minstrel, Apr 12, 2006 IP
  18. RazorWing

    RazorWing Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #18
    I deleted the three search tables which the search rebuild mod will fix, that has reduced the file size of my database from 11MB to 3MB and bigdump successfully complete the upload of the database which it didnt do before. But now i have another problem, it wont allow me to login, all tables are there apart from the three tables i deleted, all posts, topics and users are there that i am aware off. But for some reason it wont allow me to login, it generates this error message below. :(

    Error in obtaining userdata

    DEBUG MODE

    SQL Error : 1054 Unknown column 'user_login_tries' in 'field list'

    SELECT user_id, username, user_password, user_active, user_level, user_login_tries, user_last_login_try FROM phpbb_users WHERE username = 'razorwing'

    Line : 65
    File : login.php
     
    RazorWing, Apr 12, 2006 IP
  19. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #19
    It could be a phpBB version mismatch. Does the table user_login_retries exist?

    Make backup copies of your config.php and {template}.cfg files and then try running upgrade.php for your version again (I don't use phpBB any more so I may have the file names incorrect there but they'll be similar).
     
    minstrel, Apr 13, 2006 IP
  20. RazorWing

    RazorWing Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #20
    lol no matter what i try i keep getting errors :(

    Here is what i am trying to down, i am trying to move my forum from one web server to another (Fast Host to 1&1). We already know the search tables have bugs. I attempted to rebuild them on 1&1 (after deleteing the seach tables) and i was unable to login in, i applied the rebuild search table mod to the orginial Fast Host forum. It installed and i was logged in correctly but had errors when attempting to run the mod.

    Could not obtain rebuild details

    DEBUG MODE

    SQL Error : 1146 Table 'EnB.phpbb_search_rebuild' doesn't exist

    SELECT * FROM phpbb_search_rebuild ORDER BY rebuild_session_id DESC LIMIT 1

    Line : 239
    File : functions_admin_rebuild_search.php

    I tryed,
    Full Back (minus the search tables) in PHPBB (FastHost to 1&1), it worked but with errors (carnt login).
    I tryed repairing the correupted search tables on orginial source (no luck there)
    I even tryed MySQL Administrator Client, which backs up the database but i am unable to restore the database to 1&1 as it carnt find the database connection.

    Please Help :(
     
    RazorWing, Apr 16, 2006 IP