Ime, assuming the towers are not just a BS speaker with a built in stand, I’ve found them to be more dynamic above the XO than BS. The multiple/larger drivers in quality towers share thermal and power loads while(ime) delivering much more powerful midrange performance.
So imo towers aren’t just for extension.
Interestingly, I’m not sure how audyssey would set speakers to large, but then use a XO. Unless you’re referring to the surrounds? Or you had lfe+mains on. I would recommend against that personally.