Did the scrote own the wall he sprayed his paint on? No
Did he have permission No
Did it damage or deface the wall he sprayed his paint on Yes
Vandalism is an action involving deliberate destruction of or damage to public or private Property. ergo vandalism
You might not like it, but it does not make it any the less a fact.
The key terms being 'destruction' and 'damage'.
The term 'vandal' is a direct derivation from the Vandals, a Germanic tribe who were big in the later years of the Roman Empire. This is the kind of thing they got up to:
King Gaiseric landed in North Africa with over 80,000 men including Alans, Roman-Spaniards, former slaves and several Germanic tribesmen with their families. They seized lands from the local Berbers and some Romans near Tingi (Tangier), from there they overran the country and spread all over Mauritania. There was no limit to their savage atrocities and cruelties. Everything within their reach was laid waste, with looting, murders, tortures of all kinds, brigandry, and countless other unspeakable crimes, without any mercy to men, women, children, priests and ministers of god. Also they destroyed church buildings.
Not much by way of artwork, it seems. Good or bad.
This kind of behaviour explains why their name became synonymous with, say, people who slash train seats, or smash up bus shelters. People engaged in wilful and pointless destruction.
Now, you may not like Banksy's work. But you must surely recognise that whatever else, it is a work of construction, not destruction. He is making, not destroying.
What's more, and what's important, he is in the business above all not just of making art (as he, and a lot of other people, see it) but more importantly of making people think. Of changing people's perceptions. Of questioning, and challenging, their prejudices.
I don't know much about art. But if - as in the OP - it makes people at least think about things like the anti-immigrant bile that infects so much of our Murdoch/Mail-driven culture, I'd say that's a good thing. A thing, certainly, that puts Banksy in a rather different category from your average bus shelter smasher.