"It makes a big difference," Alfredsson said. "I'm really going to enjoy dinner tonight, I can tell you that, coming back the way we did."
The victory was Ottawa's second in a row following a four-game losing streak. Florida beat the Senators 3-1 at Ottawa on Oct. 22.
"We played a strong third period, which is very encouraging, Alfredsson said. It was a little bit ugly, but a very big win for us."
Keith Ballard scored for Florida, which lost consecutive games for the first time this season. The Panthers were shut out 4-0 at St. Louis on Saturday.
Tomas Vokoun stopped 29 shots for Florida.
Kelly gave Ottawa the lead at 10:20 when he poked home a loose puck in the crease. Vokoun stopped Chris Neil's wraparound attempt after a turnover behind the Florida net, but the puck stayed loose in the goal-mouth area. Vokoun stopped Jarkko Ruutu's rebound, but Kelly was able to poke the puck through.
Ottawa ended the month of October at 4-5-1, a far cry from its 9-1 start last year. The Senators end their four-game road trip Saturday night at Tampa Bay.
"We wanted to come out on the road and grab some points, Heatley said. Two out of three is pretty good so far, and we'd like to get back to .500 before we get home."
Florida led 1-0 before Heatley tied the game with a power-play goal at 5:55 of the third period. Heatley skated in from the corner of the offensive zone and beat Vokoun to the short side when he fired a quick wrist shot.
Ballard scored at 14:19 of the first period during a 5-on-3 power play just 9 seconds after Jason Smith was called for delay of game for shooting the puck over the glass. Ballard beat Auld with a wrist shot to the stick side with Rostislaz Olesz standing right in front of the net.
"You're not going to win in this league with one goal," Florida coach Pete DeBoer said. "We've got some injuries up front, but, hey, at the end of the day, you've got to score to get points in this league. We generated some chances tonight. I don't think we were desperate enough on the forecheck and around the net."
Florida almost added to the lead in the second period when Nathan Horton's wrist shot past Auld's glove bounced off the post.
Ottawa has scored at least one power-play goal in each of its 10 games this season. ... RW Wade Belak got playing time on defense for Florida, which had only five defensemen in uniform. ... Florida recalled C Shawn Matthias from Rochester of the AHL on Thursday. ... Florida LW David Booth was scoreless for the seventh consecutive game after scoring three goals in the first two games of the season.