Last winter I did hardly any rides from Christmas through to February. It took me until about August to get My fitness back to where it had been the previous November!
I don't actually want to ride much in the UK about 75% of the days in the year. I only really enjoy rides on warm, sunny days with not much wind.
If I didn't set myself targets I would probably ride less than 2,500 km a year, would get very unfit, pile weight on, spend about 160 of the 168 hours in a week alone indoors, and get depressed.
So ... I set myself targets!