The Cathedral (where I used to sing) and the area around Cathedral green including Southernhay. Mols coffee house and the Ship Inn.
The quayside as Dayvo said, nice walks around too, where 'The Onedin Line' used to be filmed. The Prospect Inn used to be good for a pint there. Walking from the Cathedral downhill past the entrance to the Chapter House and the main Cathedral School brings you to South street, crossing here takes you further downhill to the Custom House on the quay past the old Roman city wall.
There are also some interesting underground passages where they do a tour sometimes.
Unfortunately any character of the shopping area has been destroyed and replaced by an imitation of Basingstoke.