It is impossible to say from the information you are giving. However, whatever is distorting, if you run it that way for an extended period of time, it may cause damage to either the speakers or the amplifier or both, so I recommend that you always turn things down when you start to hear distortion.
If you substitute out speakers as highfigh suggests, this will not tell you whether it is the speakers or not because the different speakers will almost certainly have a different impedance curve, and the replacement ones might be easier to drive, and consequently the amplifier may not distort driving them. Also, they may be a different efficiency, and play louder at a given power output, so, again, this will make it difficult to tell which is really the source of the problem.
If you replace the amplifier with something known to be much more powerful and capable of dealing with lower impedances, then when you turn up the volume to the same loudness (measure this with an SPL meter, not with how far the knob is turned), if you have the same distortion, it will be the speakers that was the source of the problem, and if not, then it was the weaker amplifier.
Another thing to consider in all of this is possible damage to your hearing from loud sounds, so you might want to consider how loud it really is before it distorts, and it may be best to simply listen at those lower levels. If you play your music too loud, you will have hearing damage in a few years. Just do a search for loud noises and hearing loss and read up on this before you go crazy with the volume.
