I'm not really a big fan of fans
I would choose passive cooling every time! Fans are more efficient at cooling (can use smaller heat-sinks), but fans add complexity, noise, and have a relatively high failure rate (as compared to the can't-fail passive cooling). If a fan is part of the original design, then a failed fan will likely mean a failed component due to the excessive heat! Of course, you're in a different boat by adding a fan to help the passive cooling.
To my point--I choose passive cooling on my gear, and I do not add any fans to my gear. That is simply MY approach and MY philosophy. Sure, some of my gear runs a bit on the hot side. Sure, I
may lose a year of service over the lifetime of the component. But, I'm OK with that.
In general, you should be able to hold your hand indefinitely on the chassis or the heat-sink if the HS fins are external. If you hold your hand on it and must remove your hand b/c it is too hot....then that is too hot and is a problem that should be addressed.