What is depressing is not that men, too, have body-image problems (though I do find it just a touch depressing the way male body-image problems seem to focus so much on the sex organs, with the virility of a six pack coming up behind), but the way that, when discussion of body image takes place in the public space (here), men, and their penises, assume a dominating role.
I think that men are body conscious, but maybe not so sensitive to it as women are portrayed to be. We are conscious of things like belly or spindly legs and arms or no six pack, but i also think that we don't have the same pressure from the media to look a certain shape, where as women are bombarded by it in advertising every day. Advertising sets a (perceived) standard and women seem to be judged if they don't meet that standard.
Men seem to be less inclined to be so conscious, for instance, walk through town when the sun is out and lots of guys are happy to walk around with no top on, and lets face it some of them aren't exactly adonis. So maybe men feel less pressure in the body image dept to live up to.
However there is one area where the rules change, there is a (perceived ) general consensus that if you have a smaller penis you will be judged as inferior in some way. You won't match up to yet another (non existent) standard.
A big willy clearly means you are more powerful, and successful and are able to satisfy a woman from a hundred yards, where as if you don't quite match up, there may be giggles behind hands and knowing smirks. The size of manhood bears little or no relationship to what kind of person you are, you can be a great guy and have a big dick or a great guy and have a little one. I have no doubt that obnoxious people throughout the world are both blessed and cursed in the trouser dept.
The think with lycra the problem is not actually what it shows, more what it doesn't. For men, in the most part the package remains hidden and in normal clothing it is not possible to see what sort of size someone has. Bib shorts, as someone pointed out, do have a pad, and the fit of the shorts tends to squash everything so what you see isn't a definitive or detailed depiction of the penis anyway. But what the lycra does do.. and here's the bit (most) men fear... is reveal to the world that you are not packing something extra large. It gives away the fact you don't have to strap it to your thigh or tuck it into your waistband, it reveals that , in most cases, the wearer is, in fact quite normal for not having dumbos trunk stashed down his shorts. It shows that most men are average in size and the average size is less than we are made to think is "ideal".
Because of the way lycra fits it does make it feel like that area is more exposed than normal, and therefore more prone to being judged, hence why it is more of issue for men than say weight or build. To be frank, being judged by the size of your manhood is pretty much every guys worst fear, that is why it is it is such an issue, and comes up as a subject when guys talk about lycra.