As the title implies, which extension mode (20 Hz, 15 Hz, 10 Hz, Sealed) do you use?
I started at 20 Hz for music and 15 Hz for movies but after a while I just left it at 15 Hz really liked the way it sounded for music much better.
If you were going to set it differently for music and movies, I would expect it to be better the opposite to the way you originally chose to set it. Typically, with movies, the bass needs to be louder, and the sub will play louder without distortion if it is set to a higher frequency. For music (at least some of the music I listen to), setting it for about 15 Hz (or lower) would be necessary for having the deepest bass properly played.
I own 2 SVS CS-Ultra subwoofers ("old" 12" models), and I have them set to 16* Hz because that is deep enough for my music, and that way they can play louder than if I set them lower. Probably, I could get away with setting them lower anyway, but it isn't necessary, and this way I am less likely to have audible distortion in the unlikely event that I will ever push them to their limits.
My advise is to play with all of the possible settings, being sure to re-calibrate your channel levels each time you change the subwoofer settings, and decide for yourself which of the settings you like best.
_______________________
* Mine are an older model; they used to call the intermediate setting 16 Hz, the lower setting 12 Hz, and the upper setting 20 Hz. But none of these labels are precise, as the subwoofer is rated to be -3 dB at 18 Hz in the 20 Hz setting.