Actually, having a more intense thought about the situation, I have come to a conclusion of what I think happened.
1. He SMIDSYed or SMIDGAFed you and you collided with the side of him.
2. Before having a chance to speak to the driver, he disregards concern for your safety and starts giving you abuse for the collision.
3. You see this as a threat to your safety and decide the best option is to remove yourself from being vulnerable in the middle of the road and go to where you can check your bike and yourself properly.
4. The driver follows you, and as you are checking your bike, shoves you from behind whilst giving you abuse.
5. You confront the driver with (this is where I may be wrong) either questioning his driving standards or you just punch him as a retaliation from the push.
6. He sees this as a point to leave the whole situation and returns to his car whilst him and his wife continue to give you abuse.
7. The witness across the street approaches you, asks if you are okay, and informs you that he witnessed the whole thing.
I am not sure if you gave him abuse at any point (you didnt say) but by the sounds of it, you were in the right.
If not to post the video for people to critisize, then just do it so we can see you knock the pr1ck out.