Because the lights are bound to change and then you will be left in potentially a dangerous position. I too would not filter past a bus or any other heavy goods vehicle when they are sitting at lights at the front of the queue. Any cycle instructor will tell you this. Always best to stay behind, you put yourself in their blind spots, so yes, he would have been annoyed. however, he shouldn't have retaliated, gave you the finger or told you he would smack you.Why? The bus was slow to move off and I had a good speed when passing him. I judged the manouvre well and would have aborted it or apologised if I had thought it was a silly thing to do.
The footage will add clarity.
This thread makes me want to get a copy of the highway code and keep it in my pocket. i would love to have seen their faces.