Jimmy Carter left office under a hail of scorn, but his reputation has improved over the years. I personally would take this only so far. Carter wasn't a great leader. His malaise speech was a bad idea even if he never used the word malaise. Purging his cabinet in 1979 was an act of public desperation. And while he was unlucky in the timing of the Iranian revolution, he probably also mishandled the situation with the shah. He was also politically tone deaf and straitlaced, as he showed right out of the gate by primly opposing politically untouchable water projects in the West.
At the same time, his substantive legacy was really quite respectable. He was the first president to make human rights a centerpiece of our foreign policy, a stance that Ronald Reagan adopted to great effect and something that’s been a part of American diplomatic relations ever since. He managed to return the Panama Canal to Panama, a courageous but politically costly fight that could have been disastrous if he’d lost it. He helped make peace between Israel and Egypt at Camp David. He appointed Paul Volcker as Fed chairman and allowed him to begin squeezing inflation out of the economy — something that very possibly cost him the 1980 election. He began the wave of deregulation that Reagan and others extended for the next 30 years. He started the war against the Soviets in Afghanistan. And although his efforts to push energy conservation died after he left office, they look prescient now.
It's also worth taking a second look at Carter's economic policy. "Stagflation" was the word that haunted him, and the economy was hardly stellar during his presidency. Still, economic growth was surprisingly strong:

Wages declined under Carter, but starting in 1973 they declined under every president until Bill Clinton. Whatever happened, it had nothing to do with Carter:

Finally, there's inflation, the great bugaboo of the Carter years. But memories aren't always reliable. Take a look at the record:

The great inflation of the '70s started under Nixon and Ford, but Carter was the first to do anything about it. Both the much derided G. William Miller, followed by Paul Volcker, raised interest rates substantially to fight inflation—with Carter's full approval even though it promised to be politically damaging. But it worked. Inflation peaked in 1980 and had already fallen nearly three points by the time Reagan took office. Rates kept coming down after that, but it was the doing of Volcker and the man who appointed him. Reagan played no role at all.
Unemployment was probably Carter's biggest real failure. It went down and then up and he left it no better than he found it. It took the end of the Volcker recession to bring it down for good.
So that's the story. Carter presided over strong growth; got caught in the middle of a two-decade wage decline; and showed more courage in fighting inflation than any president before or after him. It all came undone at the end, but that was due to the Iranian revolution and the oil embargo, events that were outside of anybody's control.
Carter shouldn't be remembered as an economic failure. He should be remembered as a president who was dealt a bad hand but generally played it honestly even at the risk of political blowback. That's better than most.