Curtis Glencross and Rick Nash scored for Columbus, which entered as the NHL's third-best penalty-killing team. The Blue Jackets have only one win in eight games.
Nash's goal at 19:59 of the second period made it 2-2 but didn't fluster the Panthers.
"We were mad to give up a goal with a second left in the period," said Tomas Vokoun, who made 34 saves. "You have to stay positive. Let the frustration out, and it's not over. That was the beauty of the situation. We were tied going into the third."
The former Nashville netminder has won his last seven starts against Columbus and is 19-6-1 against the Blue Jackets.
The Panthers scored three times in a 3:16 span midway through the final period.
"Those two power plays we got there was a big opportunity for us to at least get a lead," Weiss said. "To get two quick goals like that was huge. The third one for a little insurance was nice, as well."
Olesz flipped a backhander from near the crease past goalie Pascal Leclaire at 10:53 on the power play for his third.
"We scored lots of goals from there, a screen and a rebound," Olesz said.
On another man advantage, Bouwmeester took a pass from Richard Zednik at the goal line and scored with a snap shot from the low slot to make it 4-2 at 13:01.
"I thought we were really loose in our own end," Columbus coach Ken Hitchcock said.
Olesz scored again a minute later to make it 5-2.
"It's no secret, there's not many pretty goals in this league," Vokoun said. "We went in front of the net and paid a price."
Glencross redirected Rostislav Klesla's shot past Vokoun for a power-play tally to make it 1-0 for Columbus at 3:13 of the opening period. The rookie has three goals in six games
Salei, who was in the penalty box when Glencross scored, tied the game 1:46 into the second with a long slap shot past Leclaire on a man advantage that carried over from the first period. The goal was his second of the season.
Jozef Stumpel hit the crossbar a few minutes later as Florida began to press with its speed.
Allen made it 2-1 at 12:17, squeezing another long shot through traffic and by Leclaire for his first.
Columbus got the puck deep into the Florida zone as the second period clock ticked down. Nikolai Zherdev fired two quick shots, and Ron Hainsey poked at the puck, which came to Nash near the crease. He swiped it past Vokoun for his 15th at 19:59 to make it 2-2. Nash has a four-game goal scoring streak.
Florida had played in five consecutive one-goal games. ... The Blue Jackets started a stretch of playing 13 of 19 games at home. ... The teams last played in Columbus in 2002.
Copyright by STATS LLC and The Associated Press
Arena: Nationwide Arena Location: Columbus, Ohio
Referees: Eric Furlatt, Justin St. Pierre Linesmen: Brad Kovachik, David Brisebois
Attendance: 13,326 (73.5% full)
Team Stat Comparison
Power Play Goals
Power Play Opportunities
Shots on Goal
Curtis Glencross (3)
(Power Play) Assists: Rostislav Klesla, Sergei Fedorov
Ruslan Salei (2)
(Power Play) Assist: Rostislav Olesz
Bryan Allen (1)
Assists: Nathan Horton, Stephen Weiss
Rick Nash (15)
Assists: Ron Hainsey, Nikolay Zherdev
Rostislav Olesz (3)
(Power Play) Assists: Nathan Horton, Stephen Weiss
Jay Bouwmeester (3)
(Power Play) Assists: Richard Zednik, Olli Jokinen
Rostislav Olesz (4)
Assists: Gregory Campbell, David Booth