Three issues commonly considered when choosing an xover point are keeping the woofer out of its breakup mode, keeping the tweeter from lower frequencies that would cause over excursion or run down into the fs of the driver, and matching directivity at the xover point for even off axis response.
This is a good statement of the general principles involved. I often get irritated by you for stating things much too broadly while ignoring the details, but occasionally you surprise me. There’s hope for you

. Of course, the devil is in the details.
In general, to perform well, a 2-way speaker with a 2nd order crossover (acoustic slopes roughly 12 dB/octave) must use drivers that have a flat frequency response and behave well for about 2 octaves above and below the crossover frequency. By “behave well” I mean woofers must stay below breakup mode and produce enough SPL roughly 30° off-axis compared to on-axis (within 3 dB), and tweeters must stay above low frequency distortion all within 2 octaves of the crossover frequency. A Linkwitz-Riley 4th order crossover (acoustic slopes roughly 24 dB/octave) requires that drivers must behave well within 1 octave of the crossover frequency. So, yes, if done right, a LR 4th order crossover does have advantages. For example, if you use a metal coned woofer with a loud breakup peak, a 4th order design can allow you to crossover that woofer at a higher frequency than a 2nd order crossover can. I think that may be why most 2nd order designs use doped paper and not metal woofers.
Many modern 2 way speaker designs employ either 2nd order hp/lp or 2nd order lp 3rd order hp (especially when tweeters are xovered at lower points).
When talking about the order of a crossover filter, it’s important to remember whether you are talking about electrical order (defined by design and arrangement of inductors and capacitors and measured by volts vs. frequency), or acoustic order (defined by the slopes of drivers combined with crossover filters and measured by SPL vs. frequency).
But the essence of the problem is that all drivers have inherent roll offs. The trick is to design the crossover to mesh with the driver roll off and or peaks to get the flattest response and provide baffle step compensation.
TLS Guy’s point about taking into account the inherent roll-off of drivers makes reference to the same thing as I did above.
A good design can achieve 4th order acoustic performance while using crossover filters with fewer parts and look like something less than 4th order electrical filters. The Dennis Murphy DIY MBOW1 speaker is a good example of this (
@everettT thanks for pointing this out). It is important to remember that this can work if you select the proper drivers, as Dennis Murphy does, but it doesn’t always work for all drivers. Most of his designs, whether DIY, Salk, or Philharmonic Audio, make use of Linkwitz-Riley 4th order crossovers, where crossover and drivers achieve 4th order roll-off slopes as measured acoustically. FWIW, I listen to two sets of speakers both designed by Dennis Murphy, one with 2nd order crossovers and the other with LR 4th order. I can tell the difference between those two speakers, but I cannot say I can tell the difference because of the crossovers.
4th order crossovers also have the benefit of a 360 degree phase shift, so there shouldn’t be any problems at the xover point between the woofer and tweeter. Wouldn’t a 4th order xover allow a lower xover point, better matching the tweeter and woofers dispersion, whilst keeping the tweeter from going down too low and keeping the woofer very far away from its breakup point, and avoid all of the problems associated with phase shift at the xover frequency?
Again you are not understanding that time and phase shift are the same thing. Do not use 360 degree phase shift, express it as time shift which you should. You would not be so cavalier about it then.
I try to accomplish all of this while trying to minimize time shifts.
To directly answer YIOF, yes, 4th order (electrical) crossover do have a 360° phase shift. (2nd order electrical crossovers have a 180° phase shift. This can easily be corrected by wiring the tweeter or the woofer in opposing polarity.)
In a 2-way speaker, the woofer and tweeter would be in phase with each other (360° out of phase = 0° out of phase = in phase). For the same reason, they would be one cycle off with regard to time.
In the past, speaker junkies loudly debated the merits of speakers that were ‘time and phase correct’. It never was clearly decided because no one could scientifically establish whether listeners could hear a difference between time correct and time incorrect speakers. Some other individuals maintained they could clearly hear a difference between them. Although this debate was never decided, it, fortunately, has grown quiet.
One of the major problems with building ‘time correct’ speakers is the expense and effort of building the physical offset between drivers so their acoustic origins are the same distance from a listener. This however ignores the real problem that even speakers with such physical alignment will still be ‘time correct’ over a narrow listening distance and angle, and over a narrow range of frequencies. At roughly 2000 Hz, one wavelength is under 2 inches. If a listener moved just a bit, all efforts at correct time alignment go out the window.