Well, I'm glad to hear that I'm not the only "non-competitive" person out there! I'm not into racing, and view the bicycle as "transportation". If someone wants to pass, fine...especially if they are nice enough to announce, I'll gladly let them by.
I drive a car or truck the same way. I drive the speed limit, and allow a lot of following distance. Often, people will fly past me, only to join the "wolf pack" up ahead, and hit the brakes. I rarely have to use the brakes on the highway...I just release the throttle, and by the time I get near the slowing wolf pack, they're going, again.
I've often wondered why someone doesn't do a thesis on this phenomenon. Why is it that when a car starts to pass another car, the car being passed will speed up? Now, 2 cars are speeding, and when they pass a third car, he, in turn, speeds up, etc. etc. Now we have these wolf packs of amateur racers, all jockeying for position, changing lanes, doing a flamenco dance on the pedals, heads spinning around, and arriving at their destination with fazzled nerves, hot brakes, and empty gas tanks. And THAT is if they're lucky! If unlucky, one person will make a mistake, and the highway will be blocked for hours until the wreckage is cleared.
Maybe, as some have suggested, it's human nature. Does it go back to the days when we had to compete for food, and the person who got there first got the food?