I wouldn't worry about the velvet. I have burgundy curtains, and a couple of folks who have been over on multiple occasions just recently figured out the color. They thought they were black the whole time. Just know that darker is better.
Yes, there are compromises when it comes to video vs audio. In my own HT setup, video won. My speakers are also too close to the sidewalls, however, not only are there heavy curtains, but panels as well hidden behind. I encourage you to treat your space, and the closer your speakers, or you, are to any boundary, the more imperative it becomes to treat those walls.
Another thing you can do is pull the mains forward, so even if they are too close to the sidewalls, they could be further from the front wall. This is in fact what I did to a modest degree.
I also have the corners trapped for bass there, and those will help you, but who can say as far as "solve". Every room, setup, and usage will vary per individual.
Perhaps of even greater importance is where your center speaker will be. The further you get it away from the floor, the better. My screen is so big, that my center actually sits on a panel. The issue with such low placement is the midbass boosting that tends to mask the dialogue frequencies. IMO, the biggest advantage of the anamorphic setups is precisely so that the center speaker can be better placed. There really isn't any other real advantage as far I am concerned.
Good luck.