I just removed the pin from the stop and sanded it down to a diameter of 4 mm (using a caliper to regularly check how much material was removed). I did this using the fixture for the pig tail mandrel on my linnisher (adapted it with a small jeweller's clamp), but I think a pillar drill would be just as good for this.
Maybe buying a new cross-stop just for this is a bit overkill, but the smaller pins have do have some impact on the offset. For most applications this might not be of any significance at all. In that case it might not be needed to buy a second cross-stop.