I've never given a thought to how other road users might interpret my wearing of glasses and to be perfectly blunt, I couldn't give a chuff what they might think. I wear them to protect my eyes and it doesn't get any simpler than that.
Perhaps I've been unlucky, but my last pair picked up a lovely deep lens scratch from a crushed drinks can that was flicked up by a car; and even without the summer insects, I'm always cleaning grit and crud off my lenses, some of which would probably end up in my eyes.
Call me cynical, but some drivers don't even register the fact that a cyclist is present on the road, with fluorescent/dayglo clothing, flashing lights et al, so to afford them the presence of mind to consider what eye protection we do or do not wear is probably asking a little too much of them.