The best I ever had for accuracy was with a shiny new Ciclomaster cycle computer, which used a barometric altimeter (last seen half way between Lhasa and Mt Everest).
Shortly after I got it, I did the Cotswold Corker (100k hilly audax), and armed myself beforehand with a list of all (or many of) the spot heights from the 1:2500 scale mapping at work. After setting the height at home, it was bang on (within 1m) until a bit over half way round, by which time the height had drifted up a bit (reading 1m high), and 2m high at the finish. The climbing was right too, being between the organiser's contour count, and one I did off 1:25,000 maps (within 15m of both).
A nice day with hardly any pressure drift, and a computer programmed by someone who had a similar view of what climbing was.
Some time later, I was on tour in Spain with it. Spanish maps are pretty poor for both contours and spot heights, so one lunchtime, when it hadn't been calibrated for a couple of days, I blagged use of someone else's GPS and tried to use that. I watched the GPS height slowly drift up and down by about 40 m 4 or 5 times over 15 minutes, whilst sitting at the cafe table, before giving up and picking a height in the middle.
With barometric altimeters, atmospheric pressure drift isn't usually too much of a problem. It's usually only about 20-30 m during a ride, it would have to be a fairly stormy day to get over 100 m, and the error in the climbing will be about the same as the drift.
What will be more of a problem is blocked pressure ports (as @Tibansman found), or wind and turbulence affecting the pressure the port sees. I have found, riding into a brisk headwind, that I can change the displayed height by up to 12 m by holding, or not holding, a cupped hand in front of the GPS. You'll also get similar, but less obvious, changes if your route direction changes with respect to the wind.
Both of these will vary depending on model of GPS, and quite possibly also on how it's mounted.
