You don't need to start anywhere other than Minehead. Head in any direction other than north and you'll find fantastic riding.
For a quick ride, go up and over Hopcott, then cross country to Horner and back.
For one of the top 20 rides in the country, head to Porlock and Dunkery Beacon (the eponymous purple-headed mountain of All Things Bright and Beautiful) - route here:
http://www.mbr.co.uk/routes/south-west-england/exmoor-gps-route-download-318776
You could also ride along past the harbour, to the roundabout at the end and simply keep going up and over Culvercliff and onto North Hill. Before you know it you'll be up on top of Exmoor with amazing views across the Bristol Channel and great trails.
Anything involving Cloutsham or Horner are also stunning, especially at this time of year.
Pop into Pompy's Cycles in town for a coffee and a million other suggestions too.
Full disclosure: I grew up in Minehead and used to ride all those on a hardtail in my youth, and having thought all of the above through, now really want to go visit my family there!