If it's for an emergency flashlight and used only for that purpose, I wouldn't leave the batteries in it. I would put them with the flashlight and label them, so that nobody looking for a set of batteries for another device takes them. As an emergency flashlight, it will probably sit for months - perhaps years - before being needed. Old batteries can leak and damage/destroy a flashlight, rendering it useless when you need it.