A year ago I got myself a road bike, and soon after some lycra shorts and cycling jerseys. I was 15 stones and was concerned because I no longer had a chin, and I couldn't see my tackle in the shower. My bosses at work all took the piss and made smarmy comments, but now they don't say anything. I am 11 1/2 stone, can fit into the clothes I kept from 10 years ago before I got married (despite my wife's many attempts to give them to charity because "they were never going to fit me again"), I can cycle to work almost as quickly as my colleagues can drive to work, I've got a nice suntan from getting out on my bike for 2 hours a day most days of the week, and my bosses daren't say a word as their bellies all hang out over their belts by several inches, and their shirt buttons strain to hold it all in. Oh yeah and I have a proper well defined chin again...
So I say don't worry about what anyone else thinks, get out there and do it and you'll soon start seeing the benefits.