"It just jumped on him," Hedberg said. "You kind of feel for the guy who wants to get a fair chance."
Kozlov and Colby Armstrong gave Atlanta a pair of one-goal leads in regulation, yet the credit went to Hedberg.
"He was our leader," Kovalchuk said. "He was outstanding. He was the best player in the game."
Ryan Callahan and Gaborik -- with his 23rd goal -- had power-play tallies to get New York even twice. Rookie defenseman Michael Del Zotto had two assists.
New York recorded a season high in shots but lost for the seventh time in eight games (1-5-2). The Rangers scored fewer than three goals for the 11th time in 13 games.
"I think everybody is mad," Lundqvist said. "I walk around angry that we are not getting the wins. I try to use it as a good thing, but of course it's frustrating."
Hedberg is 3-0 in shootouts, allowing only one goal in eight attempts. Lundqvist fell to 2-1.
New York managed only two goals in a loss to the Buffalo Sabres on Saturday, despite 38 shots.
"We've got to keep doing the right things, keep playing like we're playing," captain Chris Drury said. "If we keep getting over 30 shots, something is going to give."
Hedberg made 16 saves in the first period, 18 in the second and 12 combined in the third and overtime.
"I did see the puck well and we were clearing out rebounds," Hedberg said. "It might be a little deceptive to see the other team shooting and shooting, but if you look at Henrik, there were some huge saves on his side, too."
The Thrashers grabbed their second lead when Armstrong scored Atlanta's fifth short-handed goal this season 3:06 into the third period. Ron Hainsey cleared the puck out of his end and around the boards in the New York zone, where it bounced past Del Zotto. Armstrong snapped in a shot from the right circle.
The Rangers, who also gave up a short-handed goal in their 3-2 loss on Saturday, responded before the power play ended. Del Zotto made up for his gaffe with a circle-to-circle pass to Gaborik, who tied it 2-2 at 4:05.
Atlanta threatened to take the lead again when it had a five-on-three power play with less than six minutes remaining in regulation that lasted for 1:26. Lundqvist stood his ground, drawing chants of "Hen-rik."
Christopher Higgins had a breakaway for New York just after the teams returned to full strength, but Hedberg turned his backhander aside.
Despite racking up a large number of shots, New York didn't generate many early scoring chances. That changed when defenseman Michal Rozsival hit Drury with a pass from inside the New York blue line that sent him into the Atlanta zone.
Before Drury got off a shot, he was pulled down by Hainsey. He drew a tripping call instead of being awarded a penalty shot.
It didn't matter, because Del Zotto fired a pass from his goal line to the Atlanta blue line -- sending Callahan in alone on Hedberg. Callahan shifted from backhand to forehand to backhand before slipping the puck in at 17:10 for his seventh goal and second in two games.
"That is what is so impressive about [Del Zotto]. He is so decisive," Rangers coach John Tortorella said. "When he decides to make a play, he is going to make that play."
Callahan has four goals in eight games following a seven-game drought.
Kozlov staked Atlanta to a 1-0 lead by scoring a power-play goal with 3:19 left in the first period. The Thrashers, who have won two straight after regulation, had been outshot 14-3.
"It was very big for us," Thrashers coach John Anderson said. "We want to be on the winning track. Two games in a row is good."
Hedberg's previous high for saves was 43, done twice. ... Rangers center Brandon Dubinsky returned after missing 13 games with a broken right hand, sustained while blocking a shot at Calgary on Nov. 7. ... Kozlov has 98 career power-play goals in 1,158 NHL games. ... Kovalchuk has points in 12 of his last 17 games (eight goals, 13 assists). Thrashers defenseman Tobias Enstrom has 16 points in 18 games. ... Atlanta is 11-1 when scoring first. ... New York's 19 shots in the second period were a season high. The Rangers had 18 against Atlanta in the third on Nov. 12. ... The Rangers topped the 39 shots recorded on Nov. 5, a 4-2 win at Edmonton.
Copyright by STATS LLC and The Associated Press
Arena: Madison Square Garden Location: New York, New York
Referees: Dean Morton, Paul Devorski Linesmen: Greg Devorski, Jonny Murray