Yes he’s wrong to loiter and lecture you, BUT, he probably thought you were one of the thousands of inconsiderate, selfish, and shite cyclists out there who do zoom and weave across zebra crossings every day. He’s probably one of the growing number of peds who has little patience for a group that includes idiots who piss him off.
Similarly, most of the grief I get as a cyclist is because of the piss poor manners of many of our fellow cyclists – I don’t like it, I'm not saying that’s right, but what do you expect? We all get tarred with the same brush...