Cyclist who claim that if you do not dress up head to toe in HiViz leaves cyclists to blame for drivers not seeing them. In my experience drivers glance in a direction, look only for a car/bus shaped object and if they don't see that object pull out. I wear a Hump cover on my rucksack to keep the wife happy.
Ah I see what you mean now. I wear a yellow jacket (although it's not hi-viz) but I myself wouldn't apportion blame to anyone who didn't.
My own view is that some car drivers are very good, most car drivers are ok, some car drivers are poor and a few are downright crap and dangerous. I wear my yellow jacket because I think it may alert the poor driver to my presence, with the "good" and "ok" drivers not needing such aids. I may be wrong of course and there's probably no way of proving my wee theory either way.
Crap car drivers are another matter and I think they only miss cyclists and other vulnerable road users by luck and/or last second decisions a lot of the time... hi-viz and lights or not.