General perceptions are the problem
Firstly - cycling is scary. Wafflycat, you may be brave for doing 50 miles all on your own, apparently I am ludicrously brave for a mere 10 mile commute. Yes, I'm on my own, but it's not like there is at any point no other human being in sight (albeit in their cars).
So I'm alone, and surrounded by big terrifying vehicles which are all out to get me (my mother - on the rare occasions she cycles - rides on the pavement 'because it is safer')
Secondly - it involves getting sweaty. You get to wherever you are going and you will be unpresentable.
Thirdly - it messes with your hair. Either the helmet crushes it, or you don't wear a helmet and get all wind-tousled. Needs re-doing on arrival, either way. (My mother again - doesn't even wear pull-on jumpers so they don't mess with her hair)
Fourthly - lycra, which is of course not optional but utterly compulsary. It's not attractive that stuff which cyclists 'have' to wear. You might get seen by Mrs X over the road in your not-quite-big-enough lycra cycling shorts and then it will be the End of Times. (You could, of course, buy the next size up - but plenty of women would rather squeek in to a smaller size than buy a nicely fitting larger size).
Fifthly - it involves getting wet in the rain. Or hot in the sun. Which may make you sweaty and/or ruin your hair and/or make you look less than perfect.
Sixthly - it just takes so long (though this isn't female-specific). I mean, why would you spend 50 mins cycling to work when you can drive it in 20? Or cycle the mile round to a family member's house when you could just hop in the car?
So, all in all, going to the gym is much better - no weather consideratios; yes you get sweaty but it's deliberate and you get to fix that afterwards - it's not like you have to go out shopping looking like that, like you would if you cycled in to town; and there's no danger from other road users out to get you.
At least, they seem to be the impressions I get from family and colleagues.