What do you mean by pull one around? Do you mean turning the wheel by hand when it is raised off the floor?
If not, I advise that you stop the car on solid, level ground, turn off the engine, leave it in gear, chock the front wheels and jack up the rear of the car. Turn the wheel by hand and listen for the clunking to try to see exactly where it is.
As for what it could be, have you fitted alloys recently? Sometimes, the wheel bolts can be too long and actually catch on the rear drum mechanisms. Back the bolts off a bit and hear if the noise goes away when you turn the wheel.