Of course you will hear male voices from a sub. the fundamentals of the male voice are 80 to 180 Hz. So if you crossover at 80 Hz and this will be a fourth order filter, then the male voice will not be 24 db down until 180 Hz, it won't be 36 db down and virtually inaudible until 240 Hz.
I think there is tremendous variation in peoples ability to localize subs. When subs are crossed over at 80 Hz, I can always tell where the damn things are. That of many reasons why I advise just gently supplementing speakers with subs and no definite crossover. I think that is because I have always favored as systems with as little time/phase disturbance as possible. I think as time goes by, the late Ted Jordan will be increasingly vindicated. This is especially true if we are going to improve speech intelligibility. Separating fundamentals from their harmonics in time is a bad thing. I'm increasingly convinced we are well down the wrong path. I have good reasons for not following it.