I think there's a difference between giving money to the Legion and wearing the poppy. Both of my grandfathers were in the navy in WW2 (alas both now deceased), and I marched on Remembrance Day from the age of 8-18, once a year every year (cubs, scouts, ATC). As i said, I always put money into the collection pots, but do so without taking the poppy.
However, in giving to the Legion, I don't believe that there should be any compunction to wear the poppy. I believe it is possible to be in the collective state of remembrance that the day implies without having to wear one - the feeling of gratitude to those who lost their life, and continued to lose their lives is an internal one, not one that is denoted by what is essentially a piece of red paper.