I can find my subs at 80 Hz. I tried 60Hz to mitigate that and while that helped I still left it at 80Hz so that by turning the subs off, I kill more sound that travels in condo heaven. A 120 Hz xo is something I use for some wall mounted speakers around the tv in Mantown. There's no way for me to blend that without co-locating the sub/s and speakers. You'd think I wouldn't care with the tinnitus and all but it bugs me to no end. I understand most people share your experience but there are people out there like me.
A few thoughts, some of which may be useful to you...
Under most circumstances, your ears have a hard time localizing deep bass simply due to the length of the sound waves involved. You can locate higher frequencies easily by virtue of having a pair of ears with some distance between them. My understanding is that as the sound waves become longer and longer (200Hz is >5 feet; 100Hz is >10 feet), even the ability to detect phase differences from one ear to another becomes spotty. In home audio, this is further reinforced by having 2 or more speakers producing sound that is relatively easy to localize, with the psycho-acoustic result of perceiving that all sound is coming from those speakers.
While this tends to work fairly well in theory and in practice, there are things that can break this illusion. One example is the sub or objects adjacent to it can buzz/rattle, giving directional cues. I've also heard timing is a big factor for localization as well; if the subwoofer's contribution arrives to your ears early, you will tend to perceive that as the main source. This problem is a bit easier to deal with as reducing the distance of the subwoofer in your system setup, or otherwise introducing a small delay can improve the subjective result.