Def do it- i'm 6'3 with 36" waist, 44 chest and 15 stone so don't exactly blend in and don't fret about it at all once used to it.
The difference in your riding will be noticable (or at least i did from some cotton shorts to proper road lyrca) and if anyone says anything point out you're on a bike and it's the law.
a colleague at work in the carpark once asked if all my gear was 'strictly necessary in a work car park' I pointed out she looks like a prat in wellies and mac when walking her dog in the rain and left her to it.
The difference in your riding will be noticable (or at least i did from some cotton shorts to proper road lyrca) and if anyone says anything point out you're on a bike and it's the law.
a colleague at work in the carpark once asked if all my gear was 'strictly necessary in a work car park' I pointed out she looks like a prat in wellies and mac when walking her dog in the rain and left her to it.