I guess models differ but my RD-5500 has the bolts going in opposite ways, to achieve the 'tightening effect you describe. Looking towards the bike, the bottom jockey wheel goes clockwise and the bolt head can be seen; the top wheel goes anti-clockwise and you can see the threads of the tightened bolt (in my case with the remnants of some locktite on it).
I thought the reason for the sliding bush on the top one was more for changing/(mini-)alignment with sprocket function (ie desirable right/left movement).