Funnily enough, I had exactly the same experience on my 750 Norton.
Coming back from Solihull late one night, riding at the 30 speed limit, I got pulled, totally unnecessarily, by a marked Rover P5 that had been tailing me for about a mile. The copper was very sarcastic - "That's a very steady 30 mph you were doing there..... Sir" - so I gave him the same reply as you did.
There was nothing he could do as everything was in order, so I was on my way again.*
*I think he may have recognised me from the time I was chased out of Solihull. I lost them that time and they obviously didn't get my number.