I use both. Depends on mood, surface, bike, journey, weather, current price of sprouts.....
I have had as many hairy moments on either system, dabbing is no slower clipped in than otherwise once you've got used to SPD, and I can think of several occasions where being clipped in has an advantage.... little hops, endo turns, steep sided narrow ruts where all of a sudden you can't get a full pedal stroke in so you can toe tap or even pull up to keep going.
However, with the right shoe and flat pedal combination you will feel just as connected to the bike, the only real difference being you can't pull up from the base of the pedal stroke. You also have to learn to do proper hops and endos etc, so your skilz gets better innit.
OP, don't write off SPDs too soon, but then again don't beat yourself up if you don't get on with them.