I had a proper rigid mountain bike in the early 1990's, the reason it flew was due to very poor caliper brakes which meant you could quickly accelerate down steep hills with the brakes full on!!!!!
A rigid bike may be fine round smoother trail centres but on rougher xc riding you will take a real beating from the trail. That is before you come to a long steep off road climb on a loose trail. At that point you will understand why people want such a wide range of gears mountain biking. Many years ago a friend tried powering up a long off road hill through a forest single speed style. After five minutes he exploded and took about an hour to fully recover, saying he now understood what we were all warning him about.
Just be careful where you plan to ride or get gears