Shouldn't this be in the Cafe...?
Anyway, it's impossible for me to pick just one book.
In fiction, The Master and Margarita by Mikhail Bulgakov would be a contender, as would William Faulkner's As I Lay Dying. Also Italo Calvino's Invisible Cities or If on a Winter's Night, a Traveller, Fyodor Doestoyeksky's The Idiot, George Perec's Life, A User's Manual, Halldor Laxness's Independent People and Keri Hulme's The Bone People, Alan Garner's Strandloper, Raymond Carver's What We Talk About When We Talk About Love. I'd also have to consider Lord of Light by Roger Zelazny, Barefoot in the Head by Brian Aldiss. There's many more...
Poetry, Basho's The Narrow Road to Oku (or 'the deep north' depending on which edition you have), Tanikawa Shuntaro's Naked, Milton's Paradise Lost, Chaucer's Canterbury Tales, Seamus Heaney's North, Baudelaire's Les Fleurs du Mal, George Mackay Brown's Fishermen with Ploughs, T.S. Elliot's The Four Quartets...
Non-fiction, The Snow Leopard by Peter Matthiessen, Edward O. Wilson's The Diversity of Life, Aleksandr Solzhenitsyn's The Gulag Archipelago, Douglas Hofstadter's Goedel, Escher, Bach, Ferdinand Braudel'sThe Mediterranean and The Mediterranean World in the Age of Philip II, Christopher Hill's The World Turned Upside Down and Carlo Ginzburg's The Cheese and the Worms would be possibles...
I keep thinking of more...