Tours have got shorter, roads are better, training methods have improved out of all recognition, and equipment has got lighter and better.
As well as that up until the Lance era the top riders rode all the big classics from the start of the season to the end. No Tour contenders will do that now, they will often spend their entire season with everything specifically geared to the Tour de France and most wouldn't touch races like Paris-Roubaix of the Tour of Flanders with a bargepole. Up until fairly recently it would have been unheard of for the top riders to miss the World Championship road race, but neither Froome or Thomas bothered this year. The TdF has almost become a sport in it's own right.
I think
@User9609 was asking about year by year fluctuations, not the overall upward trend.
The most conspicuous peaks in average speed were 1971 and 1981/82. The 1971 Tour included the stage to Marseille where they arrived two hours ahead of schedule. That's enough in itself to increase the average speed by half a mile an hour. I don't know much about the 1982 Tour, but 1981 was unusual in that it started in Nice, made an immediate foray into the hills and then went almost straight to the Pyrenees, so the real GC battle started very early. Except it was more a procession than a battle.
Maybe having a closer look at some of the slower tours will reveal a stage or two where they rode particularly slowly. The 1968 event might be worth looking at. Following the disaster the previous year I believe the route was quite conservative - it was won by a sprinter/all rounder type (Janssen) - but the overall speed was quite low.
From my recollection, the first to build a season around a small number of events (basically the Tour and the World Championship, nothing else mattered) was Lemond. The modern riders look pretty busy by comparison.