My other half rides horses a lot, including on the road, and some of the overtakes she gets are as stupid as the ones we get on bikes - might have to lend her my helmentcam. It takes a particularly stupid driver to get that close to a ton of potentially-scared animal being ridden by a person armed with a whip...
As an aside, riding horses is great, and as a cyclist you'll find that you've already got the leg-strength and balance to make learning a bit easier. Give it a try if you get the chance!