Stiffness. affects the resonant frequency of the material
Yes, assuming that density and mass are held constant, increased stiffness would yield a higher resonant frequency.
Density. Affects the damping ability of the material (the ability to convert mechanical movement into heat)
Density alone is not really a dampener. Density reduces vibration only by the physical reaction of increased mass requiring more energy to excite it. But take steel for example. At almost 500 lb/ft^3, it's very dense. Compare to 45 lb/ft^3 of MDF. If you have solid steel piece with no mechanism to dampen it's oscillation, it will ring(resonate) for a considerable time and at a high amplitude within a narrow frequency band. On the other hand, this higher amplitude, but much narrower band of resonance will be less audible than the lower amplitude, but wider band resonance of MDF/wood, for example.
In a sub, you would want to use the stiffest material possible for an inner layer to push the resonant frequency outside the drivers band.
True. But a wide band sub is trivial to design to be non resonant using a singe 3/4" layer of high grade ply with bracing every 8" - 10" on every axis.
The viscoelastic layer would be to decouple the inner and outer layer of material. Essentially to contain as much of the resonance as possible within the boxh.
I am assuming you mean a constrained layer construction, using two stiff walls with the visco-elastic layer in a channel between them. In this case, it is not a good de-coupling mechanism. The method of action here is that the two stiff walls act as a shearing mechanism. The visco-elastic material that is glued to both layers, is not excited maximally on a molecular level, thus resulting in maximum conversion of molecular movement energy to a thermal energy. If you were to instead glue the two stiff walls directly to each other, and then apply the visco-elastic layer on a side(internal or external), it's molecular level movement would be far less. In this case, there is no effective mechanical impedance match to transfer energy into it on a molecular level. The only opposing force is the air on one of it's sides. Not a good mechanical impedance match. But when you trap it between two stiff plates, the one (inner) plate vibrates, then the visco layer vibrates, and the other opposing plate acts as an opposing anchor force, causing the molecules in the visco layer to move against each other with high friction. Without this opposing plate, the visco material is left vibrating mostly as a single mass, without much internal molecular friction occurring.
The outer layer would be high density (mdf) to absorb what little resonant energy was transferred through the viscoelastic layer.
I usually only recommend MDF externally if you want to paint the cabinet. Ply is superior - but it is not easy to paint and get perfect finish results.
-Chris