Really? No one has mentioned the correct answer yet?
The most important component in any sound system is...
THE ROOM!
The room is a full 50% of any sound system. All other components are just a portion of the remaining 50%/
Put any equipment you like in a tile bathroom - it's gonna sound like crap! Pay attention to room acoustics, treat the room, and position everything carefully, and inexpensive gear can sound far better than expensive, objectively better gear that's being hampered by the room it's in.
So:
#1: The Room
#2: Speakers
#3: Source Quality (I'm talking about the recording quality here, not the playback device)
#3a: Playback Device - but only if it's analogue. If it's a digital output, it's either perfect or broken, so it doesn't even make the list.
#4: Signal manipulation (room correction, EQ, listening modes, DSP, etc.)
#5: Amps