Somewhat ironically I get more abuse from other drivers when waiting on, what I consider, a safe place to pass than I ever get whilst riding a bike.
I've fairly recently started driving again, and I'd say this is true. As a driver though, I'm happy to suck it up. I'm not about to put a vulnerable road user at risk by doing something I feel unsafe just because some chump is flashing their lights at me.