the only difference it makes in my opinion is other driver's attitudes towards you. If they are in the wrong, the first thing they say is "you shouldn't be wearing those" as if to shift the blame for coming in too close (this happened ot me the other day when i had a hissy fit at a lorry driver), even though they probably had their own music blasting out. It was unlikely the lorry driver knew i was wearing them until i pulled up to his cab to have a go at him.
for me personally, i have no problem wearing them. even on full blast the wind takes most of the noise away (even with in-the-ear headphones) and i can hear traffic behind me anyway. If i hear a car behind me with or without headphones, it doesn't mean i can distinguish whether its going to hit me or not, so i really don't see the difference.
if i want to turn right, i look behind me. i don't need mirrors (if you say to a driver he has music on, he will say "but i have mirrors") as I don't have blind spots
there have also been times when i haven't been wearing them and i have heard "ghost traffic"... i thought i heard a car, looked behind and the road was clear. what's that all about then? my mate says the same.