I think I just have too many competitive colleagues and family members who cycle which has led to a bit of self doubt when I now realise that I am cycling at a fairly decent pace and needn't worry about trying to go at race speeds as I have no intention or desire to race. I guess the burning question I had in my head was 'are these guys quicker because they have carbon bikes and better wheels etc?' but now I realise that they are quicker because they have been cycling for longer and in clubs so they have the fitness and experience behind them.
As mentioned already losing some lbs will help more than a lighter bike, however after losing the weight, get yourself fit and what better way is to lug a heavy bike around up and down hills . After all that it is all about this http://en.wikipedia.org/wiki/Diminishing_returns