This is what happens then you overly complicate things - Mac Mini 3.5mm "headphone" port is also optical toslink out (at least most of recent macs are built this way)
I would jet get rid of all the electronics you have mentioned and get a mid-tier AVR - it's will do proper bass management (aka cross-over, not a mere 80mhz hi-pass) and shutoff the rest of the speakers while headphones are connected (directly) to it
So connect toslink from mac mini to avr and from avr to speakers, sub and headphones
But back to your question, if you want your headphones (amp) to shutoff the rest of the speakers then attached - your only option is to connect them using 3.5mm to rca cable headphone out port on the NAD integ amp