Your typical hipster is a iPod listening (see Arctic Monkeys, Killers etc) skinny, rakish 20-something, wearing as you suggest drainpipe jeans (with one leg rolled up), and sometimes a sturdy chain lock around waist. They ride fixed-wheel bikes with tiny narrow handlebars, garish paint-jobs, and maybe a token front brake to avoid the long arm of the law.
I rode with some of these fixed folk that you so despise a few weeks back. Thought I was in for a casual pub trip, I've never had to push so hard. Clothes do not the man make &c
. That said the roadies were probably on runs around 10x longer or more than mine.
Argh, too much information
I do my 5+ miles in jeans and manage to avoid that particular problem - although I sweat just about everywhere else 