Cotton will hold moisture next to your body, cooling you off. Hence the climber's adage, "cotton kills". Since it retains moisture, cotton will be harder to dry when you wash it.
silk is compact, cool in hot weather, adds warmth in cold weather, and dries quickly when you wash it. It's also rather expensive.