You get any group of people together and there's going to be a dynamic, which may or may not be pleasant.
I'm not a member of my local racing-focused club but have turned up for the odd ad hoc ride and more recently some TTs and they've been uniformly friendly and uncliquey. I'm sure I could find a club where they'd shun me like a leper, but that's a function of its members, not the type of cycling the club supports.
In defence of the faster club rides - I've only ever participated in a few serious groups and there's and there's an immense amount of concentration required. As was once said of US President Gerald Ford, I can't walk and chew gum at the same time, so exchanging pleasantries while hanging a few inches off someone's wheel is just asking for trouble.