I think for some the argument is for trying to make sure drivers know it's not acceptable, you let s*** go all the time and no-one ever even thinks about it and nothing changes, then that same driver knocks someone off.
The trigger point for some drivers is very low, even riding in primary to avoid some serious pot holes whilst approaching a red light, got me a beep and a "Share the road" from a driver the other day, when I asked if she saw the potholes she said no, I said maybe you should look at the road then.."
It's easy to get conditioned to it being "just the way it is" and I see some cyclists with an almost Stockholm syndrome attitude towards drivers, having a go at people who challenge dangerous driving "I get close passed all the time and I don't say anything, man up, ignore it" kind of attitude.
I think the only way is better enforcement and a true culture change which will probably only come about if more people start to cycle and the Kevin Bacon number gets lower for actually knowing a cyclist, which is probably a chicken and egg scenario as people won't cycle if it feels unsafe.
That or dedicated cycle highways with true separation, then it will probably be roadies versus casuals