It unfortunately doesn't get mentioned as often as it should, but when tightening and loosening the bit you should be applying your force in the vertical direction as shown below. You want your wrench generally horizontal, so that when you apply force to the wrench, it is pushing up or down on the oscillating mechanism.
If your wrench was vertical, then your force will be sideways on the oscillating mechanism, and this can knock your sweep movement out of centered calibration.
[attachimg=1]