I'm not very experienced but I figured I'd put my £0.02 in.
I do a fair bit of commuting on my CX bike and it's fine, I'm running Challenge Grifo tyres at 65 psi and don't get too much rolling resistance. I also do quite a bit of off road, muddy track stuff with a colleague who brings his Scott mountain bike. Basically down hill he can ride much faster than me. Without suspension my CX bike feels like it's trying to smash my back teeth out so I have to keep speeds reasonable (this could just be down to technique). Up hill the gearing on my CX bike, double, means I have to push harder than he does with his triple but I don't mind it so much, means I get a better work out.
I may be wrong but I kinda see my CX bike as a jack of all trades, it's pretty good on the roads but will never match a true road racer, and it's pretty good on the rough stuff, but will never match a true mountain bike. However, for what I want it's perfect and I rarely get limited by terrain.