I don't see a lot of point trying to upgrade a cheap bike, unless it currently has heavy steel wheels and alloys are available cheaply. Replacing a cheap alloy wheel set with an expensive one is not going to save more than a few ounces per wheel. The potential weight saving on most components is individually small, unless directly replacing steel with alloy.
Before trying to lighten the bike by component substitution, I would firstly lose the D-lock, it's mounting bracket, the propstand, and ask myself if the gadget mounts on the handlebars were actually essential. Saddles can be surprisingly heavy, and another possibly cheap way of saving weight if a different lighter design is found to be comfortable enough.
In terms of speed, on a relatively flat route, a few pounds difference in bike weight is going to make very little difference to overall performance. Anyone who thinks that losing a few pounds off the bike is going to give them the performance of a significantly fitter or stronger rider is kidding themselves. No amount of weight-paring is going to give you more lung capacity, and ultimately that is what limits rider performance. I've got various steel machines with weights varying between about 24 lbs and 35 lbs, and the lightest one on road tyres is only about 1 mph faster on overall average speed than the heaviest one on knobbly MTB tyres. If I compare the lightest and heaviest on road tyres only, and exclude the knobbly tyred ones, the average speed difference is only around 0.5 mph. Cyclists really should not get too excited about bike weight unless they ride a lot of hilly routes where they will actually notice reduced weights when climbing gradients.