As mentioned above, are you using clipless pedals or flat pedals with trainers/non cycling shoes? If it is the latter then clipless pedals will likely be better as they keep the leg 'locked' in a more stable position, and as above, things like wedges can enhance this.
If you are already clipless then I would be sceptical of the standard Velcro fastening knee support type of thing you'd pick up from a chemist. I don't really see how they would keep the patella in place. You could try something like leukotape but this has the disadvantage of being disposable.
Neither of my kneecaps actually sit in the groove, they both slide off at odd angles (first time the ortho consultant had ever seen this!). It doesn't really cause me issues but I think that is because the whole chain from hip to ankle is kept stable through the pedal stroke thanks to a properly fitting bike and shoes/pedals that work for me.