Gustavsson had 30 saves, including 12 in the third period as he repeatedly turned away the Rangers on the power play, and Toronto snapped New York's five-game winning streak with a 4-2 victory on Monday night.
Phil Kessel had two assists to add to his NHL-leading point total. The Maple Leafs scored twice in less than 2 minutes to take a 3-0 lead early in the second, then gave both goals back in another quick burst later in the period. Clinging to the one-goal margin, Toronto survived a power play in the final minute before David Steckel scored an empty-net goal with 5 seconds left.
"We had a bit of a lull in the second there when we got into penalty trouble," said defenseman Cody Franson, who scored his second goal of the season, "but we came back, and I thought locked up the third period pretty good."
The Rangers had won seven straight at home. Their only two regulation losses at Madison Square Garden this season are to Toronto, which also won 4-2 in New York's home opener on Oct. 27.
Gustavsson started in goal after Reimer returned in their loss Saturday in Boston, his first game since Oct. 22. Gustavsson has won five of his last six starts.
"We needed him to make a lot of big saves," coach Ron Wilson said. "He stood big in the situations where the puck was in tight around him."
The Rangers missed a chance to tie it early in the second period when Sean Avery briefly had an open goal after Gustavsson left the net and couldn't clear the puck. But the goalie was able to sprawl back in front at the last moment to preserve the lead.
The Leafs quickly made it 2-0. After a faceoff in the Rangers' end, Michael Sauer couldn't clear the puck, and Connolly controlled it with his skate and flicked it past Henrik Lundqvist just over 5 minutes into the period.
"We just got a bad bounce there," Lundqvist said. "I think we won the faceoff but somehow they won the battle in front of the net."
Lupul camped out in front of the goal to score on a power play 97 seconds later, with assists from Tyler Bozak and Kessel.
The Rangers made it a one-goal deficit again with two scores in 92 seconds. Artem Anisimov pulled them within 3-1 with 7 minutes left in the period, with assists from Brandon Prust and Michael Del Zotto.
Just over a minute later, New York had a 5-on-3 advantage after Bozak was called for hooking with Steckel already in the penalty box. The Rangers needed only 20 seconds to score, with Marian Gaborik making it 3-2.
"I thought we gathered ourselves and found a way to get back into it," coach John Tortorella said.
The Rangers had a power play midway through the third when Bozak was again called for hooking but couldn't shake their struggles with the man advantage. Ryan Callahan had the best chance, alone in front of the net, but his shot sailed high.
New York has scored on just 15 of 98 power plays this season.
Franson scored just over 6 minutes in to give the Leafs a 1-0 lead. Kessel, who came in tied for the league lead in points (32) and goals (16), and John-Michael Liles had the assists.
New York's Brad Richards extended his point streak to six games with an assist on Gaborik's goal. He has 10 points over that span. ... The Rangers were 3-0-1 against the Maple Leafs last season. ... Toronto players' fathers joined the team on the road trip. ... Del Zotto was slow to get up after sliding headfirst into the boards with less than 4 minutes left, but he returned.