That to me sounds like that's the clutch release/thrust bearing.
Yes it's worn.
When you put you foot down on the clutch, the master cylinder pushes fluid to operate the slave cylinder.
The slave pushes on an arm or fork in between the gearbox and clutch pressure plate that actuates the release/thrust bearing.
This release/thrust bearing pushes on the diaphragm fingers of the pressure plate, releasing the tension on the friction plate underneath it to allow it to slip.
As the pressure plate is bolted to the engines flywheel, it constantly spins with the engine so the release/thrust bearing has to spin with it, which is why it's a bearing.
It looks like a small doughnut and fits over the gearbox's input shaft and usually comes as part of a clutch kit.
You can see it here, left to right are the release/thrust bearing, pressure plate, friction plate
https://www.shop4parts.co.uk/?name=store&op=Product&ProdID=26898
It's not uncommon for them to wear and chatter or rattle with your foot off the clutch, the balls inside have started to wear.
It tends to stop chattering when you press the pedal down as you're forcing it into the pressure plate.
It's more of an annoying problem than a driveability problem as they seem to last for a while in this state, though the clutch pedal may become heavy if the bearings starts to jam or can't spin as fast as the pressure plate.
There's no easy fix, it's a gearbox out job and you really want to change the whole clutch while you're at it, so most live with it until it's time for a new clutch.
What the mileage on you car?
