Yes, but you'll need to re-centre the axle.
Or replacing the spacer with one 6mm narrower may be a neater solution, although you'll still need to re-centre it.
Edit - apologies, have just re-read your post. I originally read it as removing the lock nut. Your solution of removeing / replacing the spacer is the neatest solution.