Alright, I'll be the one who encourages someone to play with fire.
I hooked up my parents' system like that years ago, and everything still works fine. The TV only has two inputs, but they have three sources. So, I used combiners to run two of the sources into one input. As long as only one of the sources is on at a time, it works fantastic. I'm not saying that there's isn't risk, but my limited experience was with successful implementation of that approach.