I do most of my riding alone, for fitness and just to enjoy the countryside around where I live. A sort of spur-of-the-moment thing, where I can go anywhere i like at the drop of a hat. The thing with cycling, though, is that the more you do, the more you want to do. I sort of fell into doing 40 mile charity rides and 100km audaxes with small groups when encouraged by some other cycle enthusiasts, and I do also enjoy riding in company. Pub stops are a nice bonus when riding together, and I love the random cameraderie of audaxes and big mass participation rides.
The problem I find is with mixed ability levels when trying to get your mates together, so if not on my own, I tend to go out with one friend at a time and fit in with the speed they find comfortable. I once went for a 40 miler with one friend who turned up in lycra, looking supremely fit and riding a matching brand new Orbea racing bike, and another friend who turned up on a creaking old hybrid, while wearing trousers and a coat and carrying a rucksack full of provisions. One wanted to ride at about 15mph, which I could match on the level, and the other needed to go at something more like 10mph and fell well behind every time we hit a hill. Try keeping a peloton like that together and you'll probably go back to riding alone.