Yep... But when asked which to buy first, then you got my answer... as long as you have an adequate power source, then you should upgrade the speakers.
Thats in addition to the fact that the largest performance gain made is the speakers. However, with low powered amps, you got to be carefull not to clip them.