I think many of us have been through this. I am like this about flying now. Used to enjoy it, now I think about crashing. You know in your logical mind that you will be fine, but when you start feeling a bit down or nervous, logic goes out of the window and irrational fears creep in and your imagination goes into overdrive.
In cognitive therapy, there are lots of exercises that can help you get back to a more rational way of looking at things. You can write stuff down, and then write a rational response. Writing it down in black and white can really help:
'I keep thinking I will fall off the bike'
Response: I might but it's unlikely. I've ridden hundreds of miles without falling off. Even if I do, it is unlikely to be serious.
'I might have to stop riding half way home'
Response: I might, but if I feel like that I can get off and walk for a bit or ride on the pavement.
If you want to keep riding those routes it's important that you keep using the same routes where the original panic attack took place . If you can't face it at your usual time, ride when it's quiet. Otherwise you will associate that site with anxiety and the association of the feeling of panic with that route will make it increasingly difficult to ride it.
If it is at the stage where you don't want to ride at all, could you find a friend to go out with you? As other posters have said, it is very, very common to lose confidence in riding, driving, public speaking, being in crowds, and so many other things, sometimes after an incident, other times out of the blue.
Panic always subsides on it's own after a while. It's just adrenaline triggered by your thoughts..... If you relax your body and your thoughts, it will dissolve on it's own. A good technique is not to fight it (you produce more adrenaline) but to try to 'float' through it. After a couple of major operations in a short period and subsequent ill health, I suffered massively with anxiety and panic attacks for a period of a few years. Anxiety about one thing can soon leech out into the rest of your life too if you are not careful.
A great book for help with negative thoughts is Stop Thinking Start Living by Richard Carlson - usually cheap second hand on Abe books or Amazon. Really hard to type replies on the new format of the forum, so I hope this makes sense. Best wishes.