I used to think they were cute (and seeing the odd one never really bothered me) until they took up residence in my food cupboards. Then I went all out tactical nuclear and called in the services of my pest-control neighbour with his armoury of anti-mouse weaponry; poisons that require a licence to handle (the green pellets you can buy in DIY shops are next to useless. And mice are actually quite intelligent - they remember where you've put it and will avoid it second time round), and I'm ashamed to admit now, inhumane glueboards. I really hated using those. Sorry, mice.
Then I pulled my kitchen apart to find out where they were getting in. Blocked up the tiny holes (mice have flexible skeletons and can squeeze through unbelievably small gaps) with fine wire wool and that squirty frame sealant that builders use. Sorted.