While that's true your subs are still active in that range and with room gain being non linear (I think) and room modes being non linear (I'm certain) their position can still be a factor in that peak/null you have.
Yeah, I realize that there's a slope to the x-over and the subs will still have some output above the x-over frequency. But, I tried measurements with the subs alone, mains alone and subs & mains together. I only get that peak when the mains are on. The subs don't exacerbate the problem when they are on with the mains. I'm just using my Rives test tone CD right now and although it's just 1/3 octave tones, it's good enough to pinpoint that hump. I've downloaded some 1/6 octave tones and I'll narrow down the frequency response a bit further, later on.
Try for an asymmetrical placement of the subs as much as your mans will allow and try pulling the mains off the side walls too. How about taking the measurements like 1' behind or 1' ahead of your listening position and maybe moving that?
I'm quite limited in placement of the mains and the subs. Permanent placement will have to be behind the false wall. I've tried moving the mains laterally, but it doesn't seem to make much difference to the problem frequency. It's the distance from the front wall that's the culprit. I've placed the subs in the corners with the mains behind the screen, close to the frame. And, although that gives the mains a bit of distance from the side wall, they're only about 7' apart then - which is pretty close when you consider that the couch is about 12' from the screen. I've also placed them behind the false wall outboard of the screen frame, which gives me more separation, but then they're only about 7" - 8" from the sidewall. In that placement the distance between the side and front walls is almost identical, which is not a good thing, as you know. Those are basically my only options with mains placement.
I've tried placing the subs in the corners, as well as at 1/4 distance from the sidewalls and although it makes a difference, I have to say that I haven't really concentrated on fine-tuning that placement, as the problem with the mains has taken precedence. They are two different brands, which I'm sure doesn't help, but I will be building a pair of identical subs later. So, I don't think I'm going to put a priority on sub placement right now.
I expect that permanent placement will be mains behind the screen, with subs between them. That will keep the subs at 1/4 distance from the sidewalls and free up the corners for floor-to-ceiling bass traps.
As for taking measurements ahead and behind the present listening position, I don't think I'll bother with that. For one thing, moving the couch back will put it even with the surrounds, which I'd rather not do. I can't move the surrounds back, because one of them would then be in front of a hallway. I don't think I can move the couch forward, because SWMBO doesn't want to sit that close to the screen.
I might be able to sneak the couch a little bit closer when she's not looking, but not very much. Anyway, right now, the couch position isn't a big factor. I mean, that peak is 18db and you can hear it throughout the house, so moving the couch isn't going to help much - if at all. After I try some other measures, I can take another look at that.
I'll probably try REW at some point, but having looked at the guide for using it - ADHD will kick in before I get through the first page.
The Dayton Omnimic looks appealing except the price is way out of line and it's more limited than REW.
Right now, wall treatment is a no-brainer. So, I'll go with that first, then see what else I need to do after that.
Thanks for the advice.
If you can think of any other suggestions, don't hold back.