I remember going to the Natural History Museum as a kid and being awed by the building, the spaces and fascinated by the exhibits, which both scared me and sparked my imagination. I went back recently and found that they'd modernised the exhibits and a lot of it was crappy colourful plastic stuff that talks down to kids and doesn't allow them to imagine. I really do not like a lot of modern museum curation, which seems to be based on the premise that museums should be familiar and easy for kids. On the other hand, I'm glad that you can actually get in there and get around in a wheelchair these days.
As a few others have mentioned, I prefer the odd and the idiosyncratic - like the Horniman Collection, or Sir John Soane's Museum in London, or the Pitt Rivers Museum in Oxford - although the latter makes me rather uncomfortable, knowing how much was basically seized from colonized countries.
In terms of galleries, things I love vary from whole places to particular collections and painting: I love the Rothko collection at the Tate Modern, the Van Gogh Museum in Amsterdam - there's something about Van Gogh's colours that you cannot understand unless you see them for real - the Duc de Berry's Book of Hours at the Cluny museum in Paris, the Juan Miro Foundation in Barcelona, the Uffizi Gallery in Florence, Picasso's Guernica at the Prado in Madrid; Uccello's The Hunt in the Forest at the Ashmolean in Oxford. I really want to go to the KHM in Vienna to see Bruegel's Hunters in the Snow, which may be my favourite painting ever...