I believe that there is some kind of a bearing that should hold the fan up, above the motor. I could be wrong, but I'm pretty sure that if you remove the fan completely then the shaft will not fall out of the back of the motor.
The problem is that the plastic fan is only held in its vertical position by friction/rust against the metal shaft. A couple of weeks ago I was getting loud "donk" noises when driving over bumps, and it was the spinning fan hitting the plastic tray/box as it was too low. Mine was a new motor about a year ago, so it hasn't grown much rust to hold it in place yet. Presumably it had slipped down a mm or two.
I did manage to very carefully pull the plastic fan higher up the shaft so it's now clear. There was some very tentative hammering involved, similar to what you have done. However, I did it by resting the motor shaft pointing down against a block of wood, and gently hammered the plastic fan from behind via a length of 4" plastic pipe (see doodle). My thinking is that then the motor doesn't get the shocks, only the plastic does. The cores inside the motor can shatter if you give it too much shock.
If you go too high then it will rub against the underside of the bit where it mounts - the zone in the middle where it doesn't rub at top or bottom probably has about a 2mm tolerance. It's bloody awkward and there's a lot of work in doing a trial and error fit/removal to find out.
Mine now has the fan clear of the tray like yours is, but without your nylon washer underneath. So nothing is rubbing against anything - I'm presuming something within the motor is holding it up.
Mine is in about the same position as yours - basically just the bevelled tip of the shaft is showing above it.
