i guess it could work in theory, but i would be leery about depending on those rare earth magnets considering the forces involved: the spinning blade going in at an angle, etc. even though you're pushing down on a piece of material that pushes down on the insert, i'm thinking it might get bound/caught as you're pulling the blade up, relaxing your grip, getting in a hurry, etc. i could be wrong--i just don't have a good feeling about it. not to mention keeping the magnet faces clean.
i guess it's kind of cool to think about making the insert go in and out as fast as possible, but to my mind it's more of a non-issue. it really takes no time to unscrew six screws, keeping a short dedicated bit driver with the machine. it's not like you're running production and doing this dozens of times a day.