That cubby looks like it would have been a good spot for a AT screen and projector setup.
- "Given my room layout what set up do you think would work well."
There are a number of good speakers in your price range. I'd recommend matching LCR speakers. I'd also recommond a reasonably high efficiency speaker (as you are looking for some SPL in a reasonably large space).
If you don't mind a little DIY: these look promising:
https://www.diysoundgroup.com/home-theater-speaker-kits/home-theater-series/home-theater-monitors/htm-12-kit.html.
- "Given the sunken in area of the TV, would it be good to put my two front speakers in that cubby with the TV as well as my middle speaker "
I would wall mount all three outside the cubby. Use wall-mountable speakers, obviously. I'd put the center above the TV and angle it down towards the listening position. Try to limit reflections off the ceiling.
I'd also consider sound absorption/diffraction at the outside walls and ceiling at the first reflection points for those mounts.
If you had gone the AT-Screen route: they would have been just behind the screen inside the cubby.
If you do decide to put the speakers in the cubby: consider sound absorption in there.
- " If I decide to go Dolby Atmos, will I need to have the two front speakers out of the cubby to get the sound to bounce right off the ceiling?"
IMO: If you aren't going to place speakers in the high positions, don't bother with atmos (indeed: you likely don't want to anyway). nigh-upward facing speakers means you are getting off-axis sound to your ear first; followed by reflected sound. Both are bad.
- "On the back wall, would it be better to ceiling mount (not in ceiling but hanging) my back two speakers (I have a crawlspace right above that area) or should I get two stands for those speakers? "
Doesn't matter. Just get them ear-height.
- "how far back do you think the couch can be to that back wall without causing surround sound issues? "
Best thing will be to experiment.