Not sure why people regard them as vermin (whatever that actually means). Is it because they are not an indigenous species and are out-competing the Red Squirrel? Or is it because they are perceived as doing damage to homeowners' property? I get badgers digging up my lawn, does that make them vermin as well?
I don't particularly like the Grey Squirrels I get in my garden because they have been successful in the past at getting the seeds I put out for birds. But I wouldn't shoot them for that. I just make the feeder squirrel-proof and the problem is solved.