I think Carter was a victim of circumstances. OPEC doubles oil prices overnight, triggering crippling inflation - that'll put a dent into an administration that's hard to buff out. The Iran hostage crisis hurt him, but that wasn't his fault either.
Deregulation of the airline industry has been a mixed bag, but deregulating the beer industry was a resounding success, as it fostered the emergence of micro-brewing. He tried to reform health care to improve access and coverage, but it was defeated.
He brokered the Israeli-Egyptian peace agreement, which is a much bigger deal than may be appreciated nowadays. It has prevented any more major wars between Israel and her neighbours. I mean, who would dare go to war with Israel again without Egyptian help?
His major problem was his inability to "go along, to get along" and play the back-scratching game, which lost him support within his own party. It's his personal integrity that stands out for me.
Plus, he was a fellow submariner, which absolves him of all sin.