I did a similar trip last September. Bike Bus to Bayonne, then to Narbonne to pick up the BB home. 700km in 10 days - this included 2 rest days.
The first couple of days were climbing into the Pyrenees, then over the col d'Aubisque before heading down towards Lourdes, then across country to meet the Canal du Midi for the remainder of the trip. All loaded with camping gear.
I am 51, and although I have a reasonable level of fitness, I didn't train for the trip. I knew I'd be tired after the lumpy bits so planned for the second half of the journey to be flatter.
In answer to your specific questions:
This is definitely possible and you are not crazy. The Pyrenees are so beautiful, and the the views from the higher ground so extensive that you would be crazy not to venture into the hills - advice of previous poster about pacing etc is good.
I did a write up with pictures - it's in three quite big files. I could post it on this forum if I knew how to do it!
Ergle