I live in Maryland where it has been cold this winter, but not nearly so cold as farther north. I heat my home with natural gas, and compared to a previous townhouse (more than 20 years ago) with a heat pump, the difference is large. In this area, the price of heating with electricity vs. natural gas makes gas a clear winner.
But the difference in comfort levels is also large. I found that the heat pump/forced-air heating system could produce slightly warmed air, but it always left us feeling chilly, no matter what the air temperature was. Gas/forced-air actually produces warm air when it's running and is noticeably more comfortable.
I remember asking a Heat/AC guy about that, and he answered that heat pumps that extract heat from outside air barely work well in the Maryland/Virginia area without using supplemental resistive electric heating. Farther north was not really feasible unless the heat pump is the geothermal kind. Is there something new about modern heat pumps which makes them useable in Massachusetts?