1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Super Fun...

Discussion in 'Support & Feedback' started by digitalpoint, May 27, 2014.

0
  1. #1
    Well that was a pretty craptastic 22 hours. lol

    The short version of what happened is someone at the data center where the equipment is at took the liberty of moving around some of our PDU plugs... Ultimately putting multiple of them on the same electrical circuit, so we ended up tripping the breakers at the data center because the equipment was put on a single circuit and was drawing too much current for it to handle.

    That in itself is no big deal to fix, but when you have to restore about 300GB of databases to an in-memory database cluster, it's very, very slow (it's designed to be very fast once it's loaded, but not fast with the actual loading of data).
     
    digitalpoint, May 27, 2014 IP
    xXxpert, Nusaint and Riswan like this.
  2. anzilone

    anzilone Prominent Member

    Messages:
    542
    Likes Received:
    20
    Best Answers:
    0
    Trophy Points:
    300
    #2
    Really happy to see Digital Point is back.......................:).........
     
    anzilone, May 27, 2014 IP
  3. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #3
    Oh yeah... the search engine is in the process of being rebuilding all data, so not everything is searchable quite yet. Currently everything is searchable *except* for users with a user ID > 70000 and posts with a post ID > 2,900,000... but it's working on it, and shouldn't be too much longer until the last bits are searchable.
     
    digitalpoint, May 27, 2014 IP
  4. fare

    fare Active Member

    Messages:
    89
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    95
    #4
    Yeah, it was indeed a long 22 hours for me. :p

    Glad to see it's back now. :)
     
    fare, May 27, 2014 IP
  5. Brian123

    Brian123 Well-Known Member

    Messages:
    784
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    108
    #5
    Shawn, were some posts lost during this? I seem to be missing a thread....
     
    Brian123, May 27, 2014 IP
  6. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #6
    Yeah... it ended up being faster to just restore a backup from a few hours earlier, so I went that route. So anything within a couple hours of the "event" is gone.
     
    digitalpoint, May 27, 2014 IP
    Brian123 likes this.
  7. GFX2

    GFX2 Well-Known Member

    Messages:
    764
    Likes Received:
    80
    Best Answers:
    8
    Trophy Points:
    125
    #7
    Well, let's just be happy that DP is online now.
    Thanks for fixing it fast.
    Cheers!
     
    GFX2, May 27, 2014 IP
  8. p4guru

    p4guru Active Member

    Messages:
    78
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    73
    #8
    300GB restore ! :eek:

    At least you got to test your backup and recovery process :)
     
    p4guru, May 27, 2014 IP
  9. DomainMagnate

    DomainMagnate Illustrious Member

    Messages:
    10,932
    Likes Received:
    1,022
    Best Answers:
    0
    Trophy Points:
    455
    #9
    ok.. not what I was expecting to read seeing the title "Super Fun..."

    good you have the backups! That was a long downtime. I just figured you got raided by FBI and that's the last we'll see of DP.
    jk good to see it's back
     
    DomainMagnate, May 27, 2014 IP
  10. xXxpert

    xXxpert Well-Known Member

    Messages:
    604
    Likes Received:
    34
    Best Answers:
    9
    Trophy Points:
    165
    Digital Goods:
    7
    #10
    lol,
    i was feeling very bad again and again seeing my social buttons not displaying :p anyway welcome back DP :)
    Good work shawn, you are really a hardworking person, my site has only 2GB of database yet, i can understand how it feels when you have to do something with it :D
     
    xXxpert, May 27, 2014 IP
  11. FTS!

    FTS! Member

    Messages:
    412
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    33
    #11
    I'm glad DP is back online :)
     
    FTS!, May 27, 2014 IP
  12. Helge Sverre

    Helge Sverre Prominent Member Affiliate Manager

    Messages:
    840
    Likes Received:
    99
    Best Answers:
    2
    Trophy Points:
    305
    Digital Goods:
    2
    #12
    Ahh that explains the downtime today.
    Glad it's back up, good job!
     
    Helge Sverre, May 27, 2014 IP
  13. ChrisMiller

    ChrisMiller Prominent Member

    Messages:
    1,934
    Likes Received:
    81
    Best Answers:
    0
    Trophy Points:
    315
    #13
    Hopefully this doesn't happen again between July - Dec then we wouldn't have your awesome skills to solve it.

    Just sayin. ;)
     
    ChrisMiller, May 27, 2014 IP
    sarahk likes this.
  14. DomainMagnate

    DomainMagnate Illustrious Member

    Messages:
    10,932
    Likes Received:
    1,022
    Best Answers:
    0
    Trophy Points:
    455
    #14
    btw, some data got lost. Like there were several pms that I sent/received right before the crash and they disappeared.
     
    DomainMagnate, May 28, 2014 IP
  15. Blinky82

    Blinky82 Well-Known Member

    Messages:
    1,370
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    150
    #15
    Yeap. Was it hard to setup it again?
     
    Blinky82, Jun 16, 2014 IP