As far as sizes go, you have SLR, medium, compact, and ultra compact. As you drop in size, many times you'll drop in quality - especially in low lit areas as well as soft edge issues.
As far as "compact" cameras go, I've not found any better than the Canon S400. I've purchased the newer SD400 for the wife a year ago (since the S400 color and clarity was excellent). It's thinner with a larger LCD, but it just doesn't compare to the quality pics I get with the original S400.
I've never had issues with camera shake. The S400 doesn't suffer from "soft edges" as as many ultra compacts do, like the SD400, some of the Casios and Pentax models, as well as some Nikons and Sonys. I've heard the Olympia compacts are excellent choices. None of these sub $500 cameras will compare to digital SLR's, but they're not meant to. These are pocket/purse sized cameras for the novice, or a second camera for the professional. Almost all of the 4-5mp compact and ultra compact cameras produce outstanding 4x6 prints. If I was in the market again for a compact/ultra compact, I'd pay extra close attention to the quality of the flash, and soft edges. Those IMO are the toughest areas a tiny digital camera has to overcome.
Some of the newer digital cameras will do 30fps at 640x480, and a few do 60fps at 320x240 which is pretty impressive video. With a 1-2gb memory card, you practically have a budget camcorder.
DP review is a great site. Here's another excellent site:
http://www.imaging-resource.com/PRODS/S400/S40A.HTM