I'm quite a tactile person and would probably touch a person visiting me a couple of times, maybe on the arm as I showed them in and again when I showed them out.
In the course of conversation if I'm close to a person (in proximity not personally) I will often touch their arm as I chat, I don't always realise I'm doing it.
I don't mind if people touch me, men or women, aslong as it's appropriate - I wouldn't appreciate my butt being patted for example, but a hand on my shoulder is fine.
Not so long ago an older man (he's about 60) who is a regular in the shop got far too close to me while we were talking, and as I attempted to walk away he followed me and put his hand on my neck and it really creeped me out.
Oh, and just remembered at work a couple of months ago I was unpacking promotional stuff and a customer had stood on the packaging and I bent down to move it and inadvertently touched the back of his thigh.

He seemed to enjoy it though.