Purists argue that a hardtail can be used on almost any terrain. A lot depends on geometry and setup, especially as until a few years ago most XC bikes were hardtails, and tended towards fast racy bikes, with steep head angles, 71 or 72 degrees which makes a great climbing bike, but makes them hard work on descents, especially rough steep stuff. Add to that the tendency towards narrow bars, light weight components, and an arse-up head-down riding position and it was clear these bikes were made for thrashing fast smooth fire roads and bridleways. They could handle rough terrain, but they demanded high levels of rider input, and 80-100mm forks meant they could track well on uneven ground, but big hits were not within the bike's capabilities.This suited a lot of continental riding, where the hardtail market was pushing out bikes ideally suited to trekking and marathon style races. Hardtail bikes from the likes of Merida, Cube etc are prime examples. This "legacy" continues to shape a lot of entry level hardtails.
A lot of XC full sussers were being developed and marketed at the same time. The introduction of a rear shock meant that bikes were able to track the ground better, increasing traction on rough stuff, and again, short travel XC sussers are plentiful, especially amongst the XC racing fraternity. Typically these bikes had 80-100mm rear travel matched with the same forks.
UK trail riding was developing, with riders wanting more than just smooth singletrack. Trail centres were being developed and incorporating technical features which simply demanded more bike. Away from the trail centres riders were riding bridleways and trails with drops, steps, rocks, chutes, roots, etc etc. Bigger travel bikes were being marketed, with 120-140mm travel suspension, front and rear, looking to the US for inspiration, where more extreme trail riding was influencing the development of bigger hitting trail bikes. Yeti, Santa Cruz, Ibis, Turner etc were selling bikes with slacker head angles, robust frames with stronger wheels. Bikes like Orange Five and component manufacturers like Hope developed their products, and as the technology developed the concept of the All Mountain or Trail bike grew. Many current trail bikes typically have 140-160mm travel, slack head angles for descending, steepish seat angles for climbing (although that is not necessarily the trail-bike's forte) and tend to be heftier, typically weighing between 12 and 15 kg.
I have an Ibis Mojo HD, robust (very) with 160mm forks, XT drivetrain and brakes, AM wheelset with biggish tyres. It weighs just over 13kg, and I am still trying to find its limits. They are well beyond mine at the moment !
Recently the marketing push has been towards the Enduro bike, where descending far outweighs the climbing. Enduro racers are timed over the downhill sections, and climbing stages are not timed. The downhills are more and more extreme, so Enduro bikes are equipped with heavier, stiffer forks, heftier wheels and tyres, wide, wide bars and so on. They are at the heavier end of Trail bikes, with up to 170-180 travel. They borrow a lot of stuff from out and out downhillers.
The UK then has its own evolutionary branch in the long travel hardtail, or LTHT. In the UK firms like Cotic, Dialled Bikes,On One and Ragley were starting to produce hard hitting hardtail frames, capable of taking longer travel forks, with slack head angles, 67, 68, 69 degrees, which meant these hardtails were more trail, even downhill orientated. In the right hands they could do everything that a full-suss bike could do. Initially available as custom frames but now with full build options, these hardtails are a do it all, and for some go-to bike. Many are made from steel, with budget options like On One Inbred 456 and Ragley's Blue Pig being pretty heavy, but bomb-proof. The 456 can take up to a 150mm fork (the 456 moniker related to the fact it can take a 4", 5" or 6" fork. The Blue Pig is named after a famous Calderdale bridleway descent (have a look on Youtube) and their later Aluminium hooligan hardtail, the Mmmbop was named after another Calderdale descent, itself named because that was the noise you made riding down it! The Mmmbop is discontinued, but it lives on in the Ragley Marley.
Steel framed hardtails now occupy a bit of a niche market. It's no coincidence that the makers of these frames are all based in Yorkshire and the Peaks (you only have to look at the sort of terrain we ride on to understand why) , and their fanbase here is growing. Cotic takes the concept a stage further, and has two 26 inch HT frames, the BFe (Beefy, or Burly Iron) and the Soul. The Bfe is extremely sturdy Cromoly steel, with a weight penalty, but can take up to a 160mm fork, and massive levels of abuse. Its more refined stablemate, the Soul is made from Reynold 853 tubing, which can be rolled much thinner for the same strength, and weighs not much more than a sturdy alloy frame. The Soul has a ride quality that is second to none, with a geometry identical to the Bfe, but only rated for a fork up to 150mm. It is designed to ride with wide bars and a short stem, and has a sweetspot many believe with a fork around the 120mm mark.
I have a Soul. With AM wheels, a Reba 120 fork, XT drivetrain and brakes and a dropper seatpost it weighs just about 11kg. It is a joy to ride, playful, comfortable, pops and jumps, and the frame absorbs a hell of a lot of trail chatter. It is the perfect red graded trail bike, and as long as you choose the right lines it's well capable of the like of Llandegla's black run and anything that North Wales and Scottish Red trails can throw at it. It also deals extremely well with a lot of the more technical local Yorkshire and Peak bridleways that shaped its development.