I wear cycling gear on most days as I ride at least 30 miles most days. I don't when I cycle a few miles to the pub or to go shopping. Yesterday I had a go at downhill MTB so put on my bibs and then a baggy pair of shorts and a tshirt on top. I think wear what you want or what works for you. Riding the bike is more important than what you look like doing it.