I'm clueless about how to know how large my speakers need to be for a dedicated home theater.
Don't worry about size. Some big speakers sound good, some sound bad. Some small speakers sound good, some sound bad.
I have no idea even how to start - should I be looking at how many watts the speakers produce?
Not really. What's your budget?
If I'm buying speakers separately (i.e. not as a full set), is there a problem with matching them in terms of power?
Not really, that's what level matching is for. However, the best match is the identical speaker. If you are using a flat panel, or a relatively smaller PJ screen, see if you can fit a 3rd identical speaker as your center. The benefits will be multiple. Even better would be to have all three on the same plane, assuming that the plane isn't too low or high.
I'm sure it partly depends on how loud I want it, but I guess the bottom line is when I'm looking at speaker packages, how do I know whether they will nicely fill the room, or sound like a tin can in the corner (or whether I'm paying for an ability to reach a volume I'd never use....)
Thanks for any guidance!
Speakers hate being stuffed in corners. The closer the speaker is to any boundary, the more you will hear the boundary, which means the less you hear of the speaker. A corner represents three boundaries, and so the effect is multiplied. I would not fear the tin can effect so much as the muddied/chesty effect with putting a speaker SHOVED into a corner.
Also, when you want them to nicely fill the room, it's actually the room that you hear more than your speakers. This is why many experts really advise the use of room treatments. Hear less of your room, and more of your speakers.
As for how loud you need it, that's very hard to answer. That's like if I'm asking, "If I buy a nice car, how do I know if I'm paying for power I will never use?".
So, the first question I might* ask is if you would like it just as loud as it is in a theater. Then again, I have the belief that the more you treat your room, the more freedom you have with greater volumes of playback.