Ah but you won't always get the upper hand - had some guy knock some shite out of three of my mates - not expected it, walking through the village..FFS three fellas - none of them up for anything - but caught out - had they responded then it might have been different !
I've been in a big group of respectable lads - had beers, sat eating chips, but because we looked "funny" at some lad, then he'd come and challenge us - 6 lads vs one, but because we were decent folk, it was let go - the idiot walked off...
You never know what the other person is on - drugs/booze - you have no chance !