Welcome to the forum!
Just to clarify - do you want all four speakers to play the same signal?
If you are using your computer to generate the signal, then you just want a way to amplify what's coming from your computer's audio out jack to play through the speakers. There's a number of ways to accomplish that, and I'll just list a few. A multi-channel amplifier would do that, as would any receiver that has multi-channel inputs. So, if you can find a relatively inexpensive receiver with those inputs (5.1 analog inputs, for example), you could split your computer output into four channels, connect those to four of the receiver's inputs, and connect the corresponding speaker outputs on the receiver to your wall jacks.
One way to split the computer signal would be to use a 3.5mm to RCA cable like
this one, connected to two RCA splitters like
this one. Given that the computer output jack is stereo, I'm guessing that you're forcing it to play the same sound from both channels.
BTW, do you have a set of surround sound computer speakers? If so, and if they are like mine, you could use that unit to try out your house speakers - let me know if you would like some info on that.