I wasn’t going to mention the massive pile of crumbs on the garage floor. That is your garage isn’t it?
Those are old digestive biscuits he’s reconstituting with used cooking oil into power bars for late-night rides, seeing his way with home-made methane gas light. DAMHIKT
There is all manner of snobbery at play when we go out in public. All of it is a reflection of who we want to be seen as, not who we are. In recreational cycling maybe more so than in real life.
Those who berate Rapha kit are displaying the same snobbery as the Rapha wearers who look down on the OP for his choice in attire.
Wear what you want, and what expresses who you are a person. Be comfortable, show off, knit your own kit from baler twine. But whatever you do, don’t worry what other people think.
I’d choose to ride with a new group if I were you. But I’m not. If you like their company, take it with a pinch of salt. Use their snobbery against them even - Reiver-cise your cockpit; drop in to the local charity shop and score something floral for your next run.