Would you consider the NX5 an upgrade to the RS540 or is the picture on the RS540 better? Their naming convention sometimes makes it hard to figure out what level each PJ is at.
I think that's a really tough call and I will be stating opinion more than fact as I haven't viewed the NX projectors personally at all (yet).
I think that the NX models offer a slight edge in general. JVC is putting their heart and soul into making these models the absolute best they can be. While the RS540 (X790) includes the good iris to really help peak black level performance well, it is still a pixel shifter and a model which JVC hasn't been focused on for more than a year now. The NX models is the next generation from them and really what they have been waiting for as a company. Overall, I would expect the NX series to be at least a reasonable step forward with the attention to detail the JVC will provide to these products. They really want to get things as close to right as possible, and based upon their history and tight detail orientation, I would think these models deliver just a bit more than the previous series, which is just typically the case with them.
But, that's based upon my history of seeing what JVC has done. A more honest response would come from someone making that change, and I'm not aware of anyone who has done so, or someone who has made that head to head comparison.
The RS540 is their midline model of the 1080p pixel shifter, similar to the Epson 5040/5050 model, which is a lot different than the native 4K resolution which the NX series delivers. But, the NX5 is their cheapest of the native 4K models.
Given the choice between a RS540 (X790) or a NX5 (RS1000), I would take the NX5. But, since budget comes into play, the more than $1,000 price difference would make it tough for me to go with... at least until my home theater is complete.