Your problem with this unit is that it is a poorly conceived design and overall function therefore lamentable.
You have probably relied on consumer reports. I have found one
third party review from the Yamaha website with some measurements.
The review is full of praise despite evidence to the contrary. I'm sure there was a generous backhander somewhere.
Anyhow this is a servo sub in a vented cabinet. A bad idea for a start. Servo control is a form of Eq, and vented alignments tolerate Eq poorly.
This is how I get best describe this contraption. So the the spec is 20 to 160 Hz, no 3db points quoted.
So what we have if the LFE input is used is a peak to 93 db output at 90 Hz. It is 18 db down at 20 Hz. So the servo is applying a lot of boost as a vented enclosure would normally be 48 db down 2 octaves below the point of roll off. This would account for the tendency to bottom, as I'm sure there is Eq being applied below the natural 3 db point of 60 Hz for that woofer and box. Anyhow output is 75 db at 20 Hz. Not a lot of use.
Now this where it really gets disgraceful. There is a low pass filter you can engage at 40 Hz. So it rolls off at 40 Hz where the output is 82.5 db So that sleight of hand allows them to show a 3db point of around 25Hz.
So that is no sub. I don't know what you would call it, but sub it is not. I think we should have a competition here for an apt name. Bottom flutterer anyone? That's a start.
What you are reporting is exactly what I would expect from this design. It is entirely predictable.