Many years ago I was foreman at a Mazda Dealership, in those days we worked on the Mazda RX2s and 3s, the RX3s in the mid 70s were dying left right and centre, to the point you could buy a three year old one with a duff engine for £150.
These were all four choke carb models with twin distributors and contact points, before the RX7s and 8s, and being a Dealership there was no old high mileage ones.
The minor issue was "rotor tip chatter" a bit like pinking, but generally by increasing the oil metering it would largely stop that albeit with a hint of smoke as you belted away
.
The worse thing was side seal failure which gave symptoms like a head gasket on a conventional engine, coolant tank pressurising and chucking all the water out and unable to tick over below 2000 rpm (agood one could idle below 700rpm). This involved a complete strip down, the aluminium sections that the rotors turned in had twin grooves on both sides where the cast iron sections mated against and the rubber seals failed usually near where the leading and trailing spark plugs were fitted as that was the "combustion area" although the three chambers were in each rotor ( I had to look up the name for it as so long ago
epitrochoid ), hence a twin rotor engine ran a smooth as a six cylinder.
The mod supposed to cure the side seal issue was thin tin shims that were meant to keep combustion heat from the rubber side seals, hard to tell if it was a long term cure as most owners sold them after the job was done!
Assuming no water problem, a compression test may be a guide, or maybe the coil packs are tracking and firing plugs at the wrong time
I never worked on any later ones with fuel injection and electronic ignition.
If you decide to do a compression test, only take the leading spark plugs out if it is like the ones I worked on or else the compression will blow out the trailing plug hole.
I was trying to think how it could pop back as you described, similar to a conventional engine with burnt valves, as mentioned as Dealers none were high mileage hence the main issues were as I said, I suppose a badly worn rotor tip could allow ignited gases to get to the inlet port etc.
I think I would start with a compression test, then if good look towards coil packs tracking across to fire at the wrong time, these well predated ECUs etc. but maybe whatever is used to time the spark may be another direction to look.
What model and age would give maybe more of an idea.
Failing ones often had the V4 2000E Ford Corsair engine fitted as fairly compact, although the RX3 was a coupe that had a sister car the Mazda 818 with a four cylinder conventional petrol engine of 1300cc which to me was a pretty little car. They even did an automatic that was responsive and a pleasure to drive, unlike the Austin A60 with Borg Warner boxes around at the time.
When new I was impressed how smooth the power was, you could almost say like an EV, though economy was not a high point until you took into account in effect it was a high powered six cylinder two stroke running on carbs.
