I always found doing lesser miles more of a pain than distance. I assume you only really did your 4 miles in the drier days?
No need to spend a huge amount - preference is best of all, if you have something that is cheap and it works, don't let anyone tell you any different.
Don't wear too much, it's easy to get hot. A little bit of surface water isn't usually enough to warrant putting on a water proof top/trouswers imo. There are cheap ones around (as Baz above mentioned) which are small and can easily be carried everywhere in a saddle bag.
Trousers - I hate them, I have some dhb Amberleywhich aren't overly water proof and to me just aren't nice to cycle in - I prefer shorts even when it's raining as it's easier to just wipe your legs down than some wet leggings or something that you will have to put back on at the end of the day - soggy clothes aren't nice. Do you have anywhere to dry out your clothes?
Mudguards will stop surface water spraying up, so might be worth it, and some kind of overshoe (maybe time to invest in clipless if you are up for it) to keep your feet a little drier.
Considered a longer but "safer" route?