I'm guessing it's not the direct mount type of calliper, but a centre pivot type. If it is a centre pivot type, there will be a hole in the frame behind the calliper. Put a 6mm Allen key into the hole and tighten the bolt into which the Allen key locates.