I tried uploading my pic with a custom size, I tried resizing the picture from the Wordpress media, and can't seem to fix it.
HEllo, This is not just you having the issue. I experienced the same a couple of days before, WOrdpress scale function work but nothing else, I doubt if it is a wordpress bug after the update. Anyways you can try the crop option if you want a part of the image or download the image, resize it and reupload.
I also feel same type of problem & at last i resize it in Photoshop & upload again. If possible please use this technique or pm me. I will redo it as you want.
Use photoshop to resize your logo and banner. Go this way open photoshop then open your image in the photoshop, next select the layer which you want to resize, next press control + T to show transform option and then resize it as much as you want. Hope this will help you.