<SNIP>
I also installed an electric fuel pump. I didn't bother to remove the old one, just bypassed it. I had it fitted under the bonnet on the left side of the car, right beside the engine for the windshield wiper. It is kind of loud though.. Do you typically do anything about that?
It's normal to mount them on rubber isolators or "bobbins"

Search rubber bobbin on ebay. If the pump only has two bolts it may move about too much, in this case bolt the pump to a triangular metal plate and use 3 bobbins to mount the plate to the car.
Robert G8RPI.