Oh I can understand and sympathize with the upgrade itch! Oh yes indeed!
I'd caution you though that with a limited budget, you're often much better served in the long run by waiting, saving up, and only buying the highest value, "checkpoint" products that perform well above their price competition, and demand that you pay many multiples of their price in order to get anything truly "better".
That said, I fully understand the desire to just get something new and different, and wanting to do so on a budget
I'm a big, big fan of the Infinity Primus speakers, I really am. But I have to caution you that the P363 towers are not the easiest speakers to drive. They dip quite low in their impedance and they can be a problem for lower end receivers like yours because of that. I like their sound quality. I LOVE their low price. They ARE a "checkpoint" product in my book. But I'm not sure they're the best way to go in your particular case. I'd feel a lot more comfortable if you were using a more capable receiver to power them, that's for sure.
For your price, I'd be going right for some Ascend bookshelf speakers. They're "checkpoint" speakers for sure! Really, really accurate. Ascend's speakers are the sort that allow you to learn how to listen critically. They're a wonderful benchmark for comparison with other speakers. And the SEAS tweeter that's used in all the SE models is a big cut above anything close to their price, in my book. It's a wonderful, accurate, transparent tweeter
Three HTM-200 SE across the front (there's a dedicated center version) are a fantastic choice for a very wide variety of setups. The HTM-200 SE have a somewhat unique design that allows them to be wall mounted, stand mounted, bookshelf mounted, mounted in an entertainment unit, or placed close to a wall without drastically degrading their sound quality the way most speakers are affected by such less than optimal placements.
If you can place your speakers optimally, the CBM-170 SE are really a pillar of accuracy. And the big CMT-340 SE are perfect if you need LOUD output capabilities. All offer dedicated center versions. And all of the SE speakers match nicely in terms of voice and timbre.
If you just gotta have floor standing tower speakers though, and you can't swing the Ascend CMT-340 SE on their matching "tower"-look stands, check out the Pioneer SP-FS51 or SP-FS52. They're remarkably good, very inexpensive tower speakers that are a bit easier to drive than the Infinity P363 towers.
You don't HAVE to get a center at the same time. The center is the only speaker that you can actually do completely without! I've run several systems in the past with no center speaker. So long as your front L & R speakers can produce a good, convincing center image, you don't really need a center speaker. It's good to plan ahead though and buy front speakers that do have a matching center if you know you're going to want one at some point. Personally I hate nothing more in home theater than a center speaker that does not perfectly match the sound of the front L & R speakers! When a sound pans across the front, I want a PERFECT, seamless match. But I'm really anal about that
Most people aren't as bothered by a less than perfect match as I am.
As for your neighbors complaining about your subwoofer: I can't recommend highly enough that you decouple your subwoofer from the floor!
When your sub makes sound, it also physically vibrates. With the sub sitting directly on the floor, those physical vibrations go right into the floor. And now they travel as structure-borne sound vibrations all over your apartment building! This transfer of energy into the very structure of your building is the reason why your neighbors can hear bass thumping away even when the higher frequencies are blocked. Decoupling your subwoofer (ie. putting some sort of shock absorber in between your subwoofer and the floor) alleviates this problem to a large degree!
Decoupling is not a panacea for bass problems. Obviously, the bass is still traveling through the air, and you're not going to block all of its sound from reaching your neighbors. But decoupling can make a very noticeable difference and really help a lot!
I always recommend the $50 Auralex GRAMMA or SubDude riser. I tend to recommend the larger GRAMMA , even if your current subwoofer will fit on the smaller SubDude because, hey, they're the same price. And if you ever upgrade to a larger subwoofer in the future, the GRAMMA will be ready
FYI, the excellent HSU Research VTF-2 MK4 subwoofer fits PERFECTLY on the Auralex GRAMMA
. The VTF-2 MK4 is my favorite $550 subwoofer at the moment. And it is the least expensive subwoofer that I consider capable of "doing it all", by which I mean that it can play right down to 20Hz, it can play quite loud, it can play without distortion, it's accurate, it has good, fast transient response, and it has a plethora of tuning and filter options so that it will work very well in almost any room to deliver flat, extended frequency responses. Just something to keep in mind when you're ready to upgrade your subwoofer
Hope that helps!