I still think padded shorts are better than padded saddles for lots of reasons, its not just about the padding, but its down to personal preference for sure, but I would say as a cyclist who is very much on the pro lycra side of the debate, theres no way Im remotely wearing the stuff because I think it makes me slightly more aero, Im about as aero as a brick, Im not kidding myself Im making marginal watts gains from wearing figure hugging clothing for those reasons.
I wear it because its by far the most comfortable clothing Ive found on all types of distances and rides on a bike, it adapts perfectly to your body shape, not all clothes do that btw, which minimises hot spots & chafing,and its very good at thermal management, by that I mean its good at wicking sweat away when youre hot, and you only have to think about adding layers when you are cold, most of which are easily foldup able and small enough to carry around in your back pocket, and if it rains, actually it dries really quickly too, its stuff I can wear and not worry about having some gigantic salt sweat stain on.
all of which means actually so there might not be any aero gain from it, but because Im happy and feel good wearing it,even if some others feel I may look completely stupid (and Im way past the age where I care what people think I look like anymore), I am a faster rider than if I was wearing a baggy cotton tshirt and some denim shorts or equiv like some primark leggings or something (even if they were lycra based), cycling kit is designed for riding on a bike and actually I think it does a stupendously good job at it, normal everyday clothes dont tend to be and you suffer more as a result, but wear what you are comfortable with, for me it will always be the lycra cycling kit.
but people are right, Ive noted alot over the years in my sportier clothes compared to older farts wobbling to the shops in their sunday best that yes, drivers do tend to treat them alot better than me on the close pass stakes, however I tend to ride a good 2-3ft from the side of the road, theyre often so close to kerbs they risk striking their pedals on them, I think that might play just as much part as to what theyre wearing.