Is your chain too long, did you make sure you're fitting the same number of links as the chain that you took off?
You are assuming the old chain was correct. The OP says the new chain is sized correctly, but it would be interesting to know the method used to size it.
Happy to explain further but grab a cup of tea as this is a bit involved...
Short answer is that I don't think the chain is too long. If anything I think it's too short - but I could be wrong.
Long answer: I have an unusual setup, with a Deore M4120 RD & GRX 600 2x cranks. Rear cassette is 11-42 & chainrings are 46-30. So this goes over the official RD capacity which is 41t [(42-11)+(46-30)=47t]. However, I ignore the largest sprocket when sizing the chain, so treat the 36t sprocket (which is the next one down) as the largest. This gives me [(36-11)+(46-30)=41t] which is within the derailleur capacity. And I avoid going big-big.
This set up has been working fine for me, as I can use all gears except the big-big combination. The chain doesn't lose tension in the small chainring and it was shifting fine. I used Calvin Jones' method (wrap chain around largest sprocket & chainring then add 2 links).
But once or twice, I have accidentally gone in to the combination I should avoid. So to avoid damaging the drivetrain, I decided to use an extra link in case this happens again. So when I changed the chain yesterday, I used an extra link. I checked that the chain still is tensioned properly even in the small-small combination.
Then the problem as I described above began, with the gear cage bouncing, in the small ring only. OK at this point you may think that it's the extra link that's causing the issue, and so did I. So I removed the extra link and put the chain back on. But it only made the problem worse! So it wasn't due to the chain length.
Then I thought it might be due to the worn cassette. It didn't seem worn to me, but I changed it anyway just in case. And the problem hasn't gone away. And that's my crazy chain story.
