Matt Hendricks' third-period goal gave the Capitals their first lead of the night. Ovechkin had two assists and Green and Semin both had four-game goal streaks snapped, but Washington got plenty of offense from Brooks Laich, John Erskine and Mike Knuble.
Laich added his second goal of the game into an empty net with 5.6 seconds left to seal it. Laich has five goals this season. Hendricks, Erskine and Knuble have combined for five.
"It's really important when different guys score," said Ovechkin, who with eight goals is one behind Semin for the team lead. "They deserve to score goals. They work hard, they play great and they make good plays."
In a wild and physical game that featured surprise scorers on both sides, the Capitals beat the Rangers for the fourth consecutive time. Washington is 9-1-1 in its last 11 against New York and 4-0-1 at Madison Square Garden over three seasons.
Brian Boyle, who had never scored more than four goals in a season, had two to raise his total to seven, and Rangers enforcer Derek Boogaard netted his first in nearly five years, but it wasn't enough. New York squandered three one-goal leads and lost for the third time in four games.
"I thought we were going to get one at the end," Boyle said. "That would have been a great win for us. You can't fault the effort, but you have to find a way to win."
The Rangers (7-7-1) fell to 2-5-1 at home.
"It's frustrating," Rangers coach John Tortorella said tersely.
Following a Rangers turnover in their end, Hendricks, who earlier had a knee-on-knee collision with defenseman Steve Eminger, took a crisp backhanded pass near the right post from Matt Bradley and snapped a shot past goalie Henrik Lundqvist at 7:53 for the winner.
"The forwards didn't come down to support the breakdown, and it is in our net," Tortorella said. "I thought we were controlling the period up until then."
Knuble got Washington even for the third time when he cleaned up a rebound for a goal after a thrilling rush up ice by Ovechkin shortly after a Capitals power play ended midway through the second.
Ovechkin glided past Sean Avery as he surged into the New York end and then made his way around lunging defenseman Marc Staal before shoving the puck on net. Lundqvist got his stick on it but directed it in front right to Knuble, who scored into an open net at 9:18 for his first goal in 14 games.
New York nearly led again moments later, but Ruslan Fedotenko had a goal waved off because of goalie interference.
Erskine made it 2-all with his first goal in 32 games, dating to Jan. 26, but only his eighth in 338 NHL games. Erskine skated to the top of the left circle and fired a slap shot that beat Lundqvist over his right shoulder at 2:34.
Before the surprise wore off, the crowd witnessed another shocking goal by Boogaard.
The puck bounced past Sloan at the right point, and Boogaard skated alone into the Washington zone. When he reached the left circle, the fighting forward unleash a slap shot similar to Erskine's and beat goalie Michal Neuvirth over his right shoulder 41 seconds later to make it 3-2.
Boogaard, who has three goals and 565 penalty minutes in 268 career games, scored for the first time since Jan. 7, 2006, at Anaheim while playing for the Minnesota Wild.
"Hopefully, I don't get on a streak like that again," he said.
The first period was particularly rough and chippy considering these teams were meeting for the first time this season.
It started early when uncommon fighters, Washington defenseman Green and Rangers forward Brandon Dubinsky, dropped the gloves and removed their helmets at 5:17 deep in the New York zone. Dubinsky stalked the retreating Green to the other side of the zone before they got together and exchanged punches.
Boyle scored twice around Laich's goal in a first period that featured 25 shots and 28 penalty minutes. Boyle made it 1-0 at 4:02, Laich tied it on a power play at 8:52, and Boyle connected again during a Rangers advantage at 12:56 to make it 2-1.
Boyle has scored three times in three games.
"We were behind, but I thought we played pretty well in the first period," Capitals coach Bruce Boudreau said. "Sometimes a team is beating you 2-1 or 1-0 and you think there is no way we're going to come back, but I thought there was a good vibe on the bench."
The Capitals have outscored opponents 15-2 in second periods during their winning streak. ... Boyle has two two-goal games this season. His previous high for goals in a season was four. ... New York held a 28-23 shots advantage.
Copyright by STATS LLC and The Associated Press
NEW YORK - NOVEMBER 09: The net is prepped for the New York Rangers game against the Washington Capitals on...
Arena: Madison Square Garden Location: New York, New York
Referees: Justin St. Pierre, Justin St. Pierre Linesmen: Tim Nowak, Greg Devorski
Attendance: 18,200 (101.1% full)
Team Stat Comparison
Power Play Goals
Power Play Opportunities
Shots on Goal
Brian Boyle (6)
Assists: Dan Girardi, Sean Avery
Brooks Laich (4)
(Power Play) Assists: Alexander Semin, Alex Ovechkin
Brian Boyle (7)
(Power Play) Unassisted
John Erskine (1)
Assists: Tyler Sloan, Matt Hendricks
Derek Boogaard (1)
Mike Knuble (2)
Assists: Alex Ovechkin, Mike Green
Matt Hendricks (2)
Assists: Matt Bradley, Brooks Laich
Brooks Laich (5)
(Empty Net) Assists: Mike Green, Karl Alzner
Derek Boogaard scored his first goal in 235 games as he tallied for the New York Rangers. Boogaard has scored 3 career goals and Tuesday's goal was his first goal since January 7, 2006 when the movie "Hostel" was tops at the box office.