Yes, but for a different reason. And like others have already said, the difference in power consumption isn't usually large.
Passive crossovers in loudspeakers have a large inductor coil for the woofer. It acts as a low-pass filter. The number of winds in the coil determine at what frequency the filter acts. The more winds there are in the coil, the lower the low-pass frequency, and the higher the inherent resistance. This resistance, called DC resistance (DCR),can consume some power. The thickness of the coiled wire also has an effect on DCR, the thinner the wire the greater the DCR. You can find inductor coils with wire gauge ranging from as thin as 20g to as thick as 12g.
As the wire gets thicker the price goes up, and the coil gets physically larger. As you can imagine a 15 or 16g inductor coil can take up more space inside a speaker cabinet than a 19 or 20g coil, while they have the same inductance value.
If a crossover is placed upstream from the amp, it can be much smaller because the voltage and current of the music signal before amplification is much smaller. As a result, the losses due to DCR are much smaller. These crossovers are often called active crossovers because they are built to require an external source of electric power.
So, in theory, a passive crossover in a speaker can eat up more power than an active crossover, however the difference isn’t so great that turning up the volume can’t correct for it. In some cases, a more powerful amp may be needed, but this is less common.
This a generalization, and like all generalizations, you can find exceptions. In a 3-way speaker with a low crossover frequency between the woofer and mid-range, less than about 400 Hz, the woofer high-pass inductor coil will be very large, and depending on the wire gauge, expensive. This is why the crossovers for subwoofers are placed upstream from their amps. It also allows you to choose different crossover frequencies.