Something to consider...Should you have any other problems with this guy, it would be very easy to get a restraining order. If you paint a picture to the court about the hug that started his delusions, to the PI that he hired, the harassing phone calls and unspecific threats, then the court should easily grant you a restraining order.
A restraining order would be a really good tool for Law Enforcement. If he contacted you or your wife in any way, he would be in violation of the order and criminal charges would be filed. One thing you don't know is if this guy actually follows you. If he were within 100 yards of you with having a legal reason to be in close proximity, then he would also be in violation of the order.
The order would give Law Enforcement a lot more to work with should the shenanigans continue.
Just my 2 cents.
Also, maybe its time you get a Concealed Weapons Permit