Depending on the flue and the flashing you may want a vent and/or fan to prevent mould.
How would I determine whether this would make sense?
I dunno, but we had a walled up chimney, and mould arrived.
A vent was put in and a lot of work on flashing.
I would put in a fan, even if you never use it. And a vent.
You can always run the fan after rain.
Maybe you could temporarily wall it up and monitor is for moisture and mould?
And then in a few months you would know what to do?