How do I judge the total dB output of the speakers I’m listening to?
Well, I see you've done quite a few calculations below, but I would use an SPL meter. You could use this to gauge when you reach you specified output rating of the speaker, at 1 meter, using whatever signal is specified (white noise, pink noise, 1kHz test tone, etc). It would be best to do this outside or in some location where you're not going to get a lot of room reinforcement. If you wanted to, it's possible.
But yeah, I'd measure your actual sound levels with an SPL meter.
If I understand what I’ve read the dB output vs. wattage output doesn’t make sense. I’ve read that every time you double the output wattage you add 3dB of volume. Similarly every time you increase the output wattage by 10 times you increase the dB output by 10dB. (both are comparable watts to dB). Let’s use the 10dB per 10 x output wattage.
That's a very good generalization. Here's the equation:
10*log( W2/( W2 - W1) ) = P
where W2 is the "second" wattage and W1 is the "first, and P is power in dB.
So, if you doubled your power from 5W (this will be P1) to 10W (this P2), you get
10*log ( 10 / ( 10-5 ) )
= 10*log ( 2 ) ----- note here that the stuff in the parens just becomes the ratio of the increase in power, in this case it's doubled.
= 10*3.01029 ~= 3 db
which is your 3 dB increase for a doubling of power.
Similarly, for a 10x increase in power, you'd get:
10*log(10) = 10 dB
If I assume a 0 dB volume level means 1 watt (with a true output in dB equal to that of the speaker sensitivity) the numbers don’t work out.
0 dB on the knob doesn't really mean 1 W at the output. I usually tend to think that it means there is neither amplification nor attenuation applied at the preamp stage. I've measured a few preamps from input to output, and at 0 dB, I get pretty close to a pass-through signal. Now, since that 0 dB number is applied to the preamp stage, the power output at the speaker terminals will be dependent on the amplifier attached, so it could be anything.
If you were actually to do the (rather non-scientific) test I suggested initially, you'd find that a 1W output at the amplifier will actually correspond to a much lower number on the volume knob display. I don't know what it'll be, but it'll be less than 0 dB on the display.
Is the 10*output wattage giving 10dB too much of a generalization?
Nope, it's exactly correct.