I could not say what the manufacturing costs are, but when buying raw drivers and raw crossover parts, the prices vary considerably. There is one indication of greater expense that is obvious, and that is having greater mass, as that means more materials are needed. And obviously, making something to tighter tolerances takes more care and more expensive equipment. But that probably only gets us up to justifying a few thousand. I think there generally is no justification for speakers costing over $20k.
In the case of my Apogee Stage speakers, there is a very large coil in the crossover, several inches across, and the crossover contains many more components than one typically finds in a cheap two-way crossover. So there is more actual cost to the crossover than is often the case.
Here is a picture of a Stage crossover:
http://www.apogeespeakers.com/projectstech/jeff_blazey/jeffs_stage_crossover_03.jpg
The cardboard with the writing on it covers a large coil.
The Stage, though, originally retailed for about $2k/pair, and, according to what I have read online, quickly climbed to $3k. I do not know the final retail price when they stopped making them. But we are not talking crazy expensive, just more expensive than most people want to spend.
Another thing to remember is that retail price usually involves the manufacturer making a profit, as well as retailers. I would expect actual costs to make are often about 1/4 retail price, but that is a guess (obviously, it is going to vary in practice, and manufacturers who sell direct don't need to include the same profits for retailers in the retail price). But maybe someone with some real numbers will be able to chime in on this, but I doubt it, as there is good reason for a manufacturer to withhold cost information to consumers, as people are often unreasonable and want to not pay what it costs to keep a retail store in business, or for anyone to make a reasonable profit on the process. Also, shipping speakers, particularly large and heavy ones, is also going to add to the expense, sometimes quite significantly. And the shipping of the speakers to the dealer is included in the retail price of most speakers.
With the crazy expensive speakers, I would guess that it is far less than 1/4 retail price to make them, but they probably put "magic" wires and other nonsensical things in so they can pretend one is getting value for one's money.
As for the cabinet costing a lot, I have seen gorgeous speakers at prices that are not insanely expensive. Aurum Cantus comes to mind:
http://www.kellsieavdesign.com/products/Leisure2SE.htm
$1500/pair for the U.S. version (which costs more and is prettier than the "international" version), and they look as beautiful as just about any speaker I have seen at any price. The connections are nicer than on my more expensive Stage speakers, and the finish is nicer as well. They are made in China, and the Stage speakers were made in the U.S., so that probably explains some of the price differences, though one also gets to pay shipping to the U.S. (included in the retail price, as the retail price is what one would pay at a dealer in the U.S.). The Stages sound better, but they are less convenient in many respects, including placement and room required for them, and a 3 ohm nominal impedance. The Aurum Cantus Leisure 2SE (U.S. version) is an easy 8 ohms, normal sensitivity, and a nice compact speaker that takes up far less real estate. The treble, in my opinion, is just as good in the Leisure 2SE as the Stages, but the midrange and bass are better in the Stages.