I completely agree with you. More often than not if I have somebody right on my back bumper when I am travelling at the speed limit (or what I think is safe and comfortable under the circumstances) I just look for a space to slow down and wave them past.Riding, and falling off, motorcycles when I was young taught me an awful lot about reading road surfaces and trying to read the intentions of others on the road.
Being enthusiasts we are all interested in our vehicles and in trying to drive them well and sympathetically. One of the worst mistakes you can make is to assume that the person driving any of the vehicles around you has the same interest. Most do not. Of course they aren't "trying" to have an accident with you - well not many are - but they are often not engaged, not concentrating, don't have the experience and totally lacking in imagination or the ability to foresee the consequences of their actions/inattention. Give them all room. Don't "compete" with their silly games - if someone is trying to cut you up just drop back, If someone is aggressively pushing your back bumper just let them overtake when safe and they can go off and have their accident with someone else. Most of all though, whilst you may be thinking "what a rude prat" don't let it get you excited or upset. Life's too short to use some of it up being annoyed by things like this.
What really annoys me regularly is when you are at a junction, waiting to turn right or similar and a car with heavily blacked out windows appears. I really miss not being able to see the other driver's eyes. I would love to be a copper under those circumstances.