jcroasii, you ask an extraordinarily difficult question(s).
I am still scratching the surface myself.
I think the fastest learning curve to understand how someone derives the best circumstance/compromise of design for a given budget, size, and aesthetics, is to buy speaker building books, hang out with builders, and start making them.
It just looks like a box with drivers, but damn there is a ton of math behind it.
The size limitations, whether large or small, are often determined by the design itself. Then the driver choices may determine which designs are even possible to begin with.
The material itself is probably one of the least important things to consider, in the grand scheme.
The goal is linear response, that you can then tailor to taste with EQ, perhaps.
For some, having rather linear response at greater offaxis angles is of very high importance.
Passive xovers are naturally complex, and so one would hope to keep it away from the speech discrimination band, and that would be at the very least as high as 2khz for tweeter, if not double that.
Just for diffraction (which makes certain frequencies abnormally louder than others when "freestanding"; different design parameters for infinite baffle aka inwall), there is a lot to it, and I just learned this stuff very recently. There may dimples on your baffle, or having a rounded sloped baffle, or at least having rounded edges, with drivers asymmetrically and strategically placed so that diffraction effects are evened out so to speak. See, this is just one thing of many other things.
The speaker is basically comprised of three things, the drivers, cabinet and xover(s). For some experts it is the driver choice that is paramount, and the lowest common denominator of how good the speaker can sound. Then others would tell you the cabinet is the most expensive part of a speaker purchase. Others yet would tell you when you listen to a speaker, you are listening to the xover.
It's really an overall design of high complexity at times.