Could be either, depends on too many variables to comment on who is in the right, there might be just a few mph or a few yards between the correct decision to pass or wait. In this case the driver appears to have taken the safest choice and aborted his overtake, well done of him.
If a driver is obviously going to pass you, and he has room to do so, then I reckon you should not signal until he is past. For what its worth I never signal in such situations, I will pick my spot and let my road positioning make my intentions clear.
Very good of the driver to take note of your signals and act upon them.
