One body of generally weird music I always like to raise an awareness of is Scandinavian folk fusion. There are a number of groups and one label Northside in particular that are easily accessible.
Varttina: A psycho girl band - originally more folk, tried several different styles on the way, and their last few albums are worth more than novelty. For sheer entertainment though nothing beats the album "Aitara". The first song alone, the first 5 seconds even, are worth the cost. At least go to Amazon and listen to a clip.
Garmarna: Think slightly Swedish-flavored Massive Attack. I personally like international music just for the relief of being able to appreciate the music without trying to decipher lyrics and meaning. Sometimes that ruins what was a good song musically, you know?
Gjallarhorn: Pagan/tribal music with great didgeridoo and rhythm.
Hedningarna: A bit heavier than the others mentioned. Hippjok is a good album, great pounding drums and stuff especially on the club beat strange last piece.
Hurdy Gurdy: The entire album consists of music played on the Hurdy Gurdy, with the rhythm section built from modified samples. Cool stuff.
Hoven Droven: Another more rock-flavored band, except the album Hippa which is some of the best mellow chillout music I know of. The last track makes everyone feel happy.
Try them and get hooked.