Brake upgrades are usually my first choice. And, I NEVER skimp out on tires!
All the power, speed, handling in the world is nice, but you better be able to stop when you need to!
Drilled/Slotted rotors, performance pads, braided brake lines, will do wonders! I have found I can get this stuff delivered to my door from rockauto.com, and not really much more cost than the economy crap from the national auto parts stores. And, the performance parts have a longer life too, so you really come out ahead on the $ in the long run.
And, brake upgrades are a big driver to start looking at bigger rims, to handle bigger rotors.
EDIT: I treat my mountain bikes the same. Bigger rotors, Shimano Hydraulic Brakes, tires run from ~$50 to $100 a pop!