I think the Galaxy has two fixing points on the seat stays.
I'm surprised to read of this problem. I've been using rear panniers alone for years and never had any problems. This is on various frames, most recently an elderly Holdsworth 531 tourer (dating from the 1980s) which is very much like the Galaxy I believe.
Probably riding position counts for a lot here. Certainly a 'forward' position will help, with the saddle set a bit higher than the
top of the handlebar. And the saddle should be at the correct height (try the '109' formula). And certainly not too far back: on mine, I have a seat post with very little offset between the post and the saddle clamp, and have the saddle slid as far forward in its rails as it will comfortably go. If that makes the handlebar too far back you need a longer stem.
Lastly, is your frame the right size for you? There are various formulas for calculating the ideal frame size for your height. If you've got it wrong, time to start taking the hormones...!