You should buy a stem with the same diameter as the handlebars.
26.4mm stems are designed for 26.4mm handlebars, not to give an 'easy' fit on 26.0mm ones.
Using a 26.4 stem with 26.0 handlebars is a recipe for annoying handlebar slippage and over-tightening the stem clamp to try and compensate. (DAMHIKT)