It's not so much as changing the tyres to effect the weight (it won't really anyway) - if you do that then unless you constantly change back - the thin tyres will be rubbish as soon as you go on even a fire road with a hint of mud. You said you 'might' do some off roading? well if you ain't sure, I'd get a road bike or cyclocross for the commute and save up for a decent second hand MTB for some rough stuff.
Yes it certainly will make it faster rolling - 100% correct there but I was just wondering about when you did a bit of off road? - this time of year especially you'll be all over the place on slicks. I do know a couple of lads who have bought cheap second hand wheelsets with knobbly tyres on and just change the wheels to suit? but it's your shout obvs. If you think you are mostly going to be on the road then yeah, stick with slicks/semi slicks.