I think you are right about the layers/warmth being the deciding factor. I made a lucky choice before leaving home to wear my heavyweight Merino jersey, by Strood it was damp around the hem and down the front but I was still warm. This was in contrast to many others who were shivering away in the hall.
Merino was a good choice. I'm going into geek mode here but given the Whitstable experience and since we are going into winter some may find this information useful.
When considering clothing, fabrics rate as follows for warmth:
1. Wool
2. Artificial wicking fabrics.
...
1000000000. Cotton
Wool performs well because nature evolved it to protect an animal from the elements. It protects you from both the heat and the cold. A woollen garment can absorb 30% of its mass in water and still feel dry to the touch. The reason is because the wool fibre keeps its shape when it gets wet. This allows the garment to maintain the spaces between the fibres and it's the spaces between the fibres, not the fibres that keep you warm. Wool still has insulating properties when it is soaked. Wool also has good wicking properties and dries fast. Unfortunately not as fast as nature intended because modern processing washes out most of the natural lanolin oil that you find in raw wool. It's the lanolin that promotes the wicking. Washing detergents specialised for wool will add some of the lanolin back so it is worth using them
Specialist artificial fabrics perform next best but performance varies. You get what you pay for. Also, the fabrics tend to specialise and are either designed to wick or insulate. Wool does both.
AVOID COTTON! Yes it's natural fabric and great for casual wear but it will land you in trouble if you get caught in bad weather for any extended period of time. I'm not kidding, cotton can cause hyperthermia. Here is why:
Nature evolved cotton to carry a seed in the wind and we are using the fibre for an adapted purpose. When it gets wet the fibres collapse, which is exactly what nature intended. When the fibres collapse they stick to your skin and increase the wind chill factor. Wet cotton clothing will make make you colder than wearing nothing at all. If you want evidence of this, go to a wet t shirt competition.
Cotton also absorbs water like a sponge. Great for towels but not for clothes. Cotton clothes becomes heavy and chafe. If you have walked any distance in a pair of wet jeans you will have had first hand experience of this. Not pleasant.
When cotton socks get wet they will make your feet cold and as added bonus chafe your skin and give you blisters (and sports manufacturers still promote cotton as the main ingredient in their socks). Wool or coolmax socks are a worthwhile investment.
Who was it who said there is no such thing as bad weather, just bad clothes?
Wool is unfortunately expensive but its a worthwhile investment. You can't beat woollen base layers and you can't beat woollen socks.
Switch geek mode off...