I'd suggest reading
the Wikipedia entry. The explanation is pretty good, and there are pictures.
Class A is going to be the purest, and least efficient. In general, it uses a single transistor to amplify a signal (yeah, there will ultimately be more than one in the entire signal path, and amplification can occur in stages, but each transistor will amplify the signal itself). Yes, it's kind of "on all the time" in the sense that it's
biased high enough to handle both positive and negative swings on the input signal. "Biasing" is on all the time simply means that there's a current flowing through the transistor or tube when there is zero input signal (but that current is not alternating at that time, and there is no sound). Properly designed Class A amplifiers will work only in the linear region of the transistor, which is a good thing. As transistors approach their non-linear regions, the output signal will be distorted with respect to the input signal. Class A amps are the least efficient, and a lot of the bias current, whether in use or not, is translated to heat.
Class B amps only use transistors in one half the range of the input signal. That is, one output transistor is used to amplify the positive part of the signal, and another transistor is used to amplify the negative part of the signal. As such, they can both be biased to be "off" when not in use; they will only be on if there's a signal coming in. The two halves of the signal are then recombined to create amplification of the entire signal. The problem here is that there can be crossover distortion where the signals are recombined (about the 0 point). Class B amps are not well suited for audio applications. They are more efficient than Class A or Class AB amps.
Class AB amps play some tricks to bias the transistors slightly on around the 0 point to eliminate the crossover distortion. They are more efficient than Class A, but less efficient than Class AB, and they are very common and well-suited for audio applications.
As to whether or not differences can be heard, you'll have to investigate for yourself. I'm not one to say that all amps sound the same, because there are simply too many variables to consider. There are definitely differences from an electrical engineering point of view; therefore, there's definitely potential for sound quality differences.
Good luck!