Having gone from Alu wheels to carbon wheels and the same braking surface on each wheel, I haven't found a big difference between the two. It is October since I used that bike though.
I also doesn't use it in the wet, so can't comment on carbon breaking wet weather performance.
This is with SRAM red calipers and appropriate pads for each braking surface too.
Any finally I live in suffolk, so it's not like we have any proper descents to really test the brakes on over a long period...