I don't have an impact driver but I could seriously use one. Loud? Sure, but I constantly use hearing protection anyway with all my other tools, so this one would be no different.
An impact driver's main benefit is the way the force of the motor is transmitted to the screw, it is done with lots of small, but powerful impacts. Because there is a minute interval between each impact, the strain on your arm is very small but it still gives a lot of torque to the screw, more than is possible with a normal drill. So not only can you drive bigger screws, but it is more ergonomical also.
A normal drill on the other hands, exerts a constant force on the screw. And with the physics law of action = reaction, that means the force the screw gets, your hand also gets. So doing a lot of heavy screwing can get very tiring for you, but, sometimes it can even be quite painful. It doesn't happen often, but if a large screw binds, the drill can really hurt you by hitting or twisting your hand forcefully. Not so with an impact driver.
Now if you mainly use small screws up to 5 cm / 2'' long, you won't notice the difference so much, but you really do with the bigger ones.