...Wal-Mart, K-mart, Target or any other music retailer near you...I know Rolling Rock makes its' brew down there...
Many classical labels have budget lines which are usually compilations of a specific type, say arias, overtures, marches or segmented by era or instrument...some even packaged as mood music...and there are others like the X amount of "best loved melodies in classical music" that sorta' thing...Also SONY (I think it used to be a Columbia property) has a series called "Great Performances", basically classics of the classics...noteworthy performances from a bygone era performed and conducted by some of the greats in the field.
If you are really serious about getting into classical, do a Google on "the basic repertoire" as there are a number of sites that can aid the novice in making smart choices in beginning a collection and understanding the genre...here's one link:
http://www.classical.net/music/rep/top.html
jimHJJ(...aside from porn and bad recipes, this is one case where the net can actually be worthwhile...good luck and good listening...)