since you already have the Azur, most definitely go for the headphones now. See how they work with the amp. You might be surprised.
There is a lot of ego-centric hoopla in this field and sometimes it costs a lot of money only to learn later that you didn't need to.
Naturally, headphone jacks don't send out the same power that the speaker outputs do. they ar e intentionally made that way. Headphones can't handle that much power. Headphones can only handle a certain amount of power and that's fine. Headphones don't have to fill a room with sound, only your ear canals.
If you find you don't like the sound with your Azur, then buy a headphone amp, but make sue you can return it if it doesn't live up to your expectations.