Do I owe him something? Is it punishment having someone made accountable for something they did?
I don't think so, you might.
This is both interesting and revealing but let's get rid of the cyclist v motorist war nonsense. My actions would be the same regardless of my mode of transport; I walk, I cycle, I ride a motorcycle, I drive a car.
Now, back to the matter of trust. No matter what the driver thinks he's got from me by settling in cash (and bear in mind he's avoided paying for the injuries and the damaged clothing), I owe him nothing.
When I took out my motor insurance, part of the T&Cs is that I must notify them of any accident I am involved in or any claim made against me. It's also a condition that I do not settle a claim myself or admit any liability.
Come renewal time, I would have a duty to tell any new insurer of past accidents or claims. This is part of a contract that I am bound to honour. The same will apply to this driver.
But your position appears to be, and correct me if I'm wrong, that this driver should renege on his responsibility, breach his contract, and be allowed to obtain further insurance by dishonestly representing himself; a driver who has also failed in his legal obligation to notify the police as soon as reasonably practicable that he was involved in a 3rd party injury collision.
With all of that taken into account, you say that I am the one who cannot be trusted?