We all have different ideas and some stay warmer than others. I have never found a way to stay totally dry and so I aim for damp and warm. You can make your winter kit as cheap or expensive as you want. My expensive piece of kit was a Gore rain jacket and it is brilliant.
I use this long sleeve base layer from Planet x. They are so good, I bought three. They wash well and dry quick and keep me warm.
https://www.planetx.co.uk/i/q/CLPXLSBL/planet-x-pro-365x-long-sleeve-base-layer
Aldi make a cycling soft shell jacket and I think it is one of the best pieces of cycling kit I have got. When you can get them. It is has a thin fleece inner and keeps you warm.
http://road.cc/content/review/194660-aldi-men’s-softshell-cycling-jacket
Polar buffs are great in winter. One round the ears and one round the neck. But the original buffs are hugely expensive for what they are. Make your own or look out for them n Aldi, Netto Lidl.
Arm warmers and leg warmers are great.
I could not use shorts in winter. I use bib tights with leg warmers.
Feet and hands are what seem to suffer most. On my hands I have sealskinz winter gloves and camouflage overmitts off
ebay for about a tenner. The best tenner I have spent.
On my feet. Middling marino socks, sealskinz socks over them and neoprene overboots.
You can even put your feet in plastic bags before putting your shoes on. It will help keep them warm. But they will get damp.
I am usually riding about 40km a day. Often a lot more and I always stay warm, except in freezing fog.
But layers is the answer. I am on 4 layers on top at the moment but I can ride all day in any weather.
Eventually, you may find you have to pay a bit more. For Gore and sealskinz for example. But I think its worth it. Even though my first reaction is always "How much"?
As Colin said. Mudguards can help a lot. They help keep your feet dry and that prevents windchill.