(T) Bizzare power steering pump problem?

Popped a new power steering pump on to the Teapot. All looked fine until the car was started, whereupon the new pump's spindle (on which the pulley wheel is fixed) immediately worked its way out of the pump with the centrifugal force! Am I being thick, but how is the spindle supposed to be retained in the pump when its just a plain steel shaft with a pinion on the inside end? But also on the old pump, the spindle seems to be retained in the pump and does not come out??? As always any advise gratefully received...