Broussard's three-run shot made it 5-0 and Cliff Lee and two relievers made the lead stand up.
"I think the home run really hurt," Orioles manager Lee Mazzilli said. "It changed things real quick. It puts you down five right away, and you're not out of the first inning yet."
The Indians took three of four from Baltimore to wrap up a 5-2 road trip and improve to 12-5-1 in their last 18 series.
Travis Hafner went 2-for-3 with a homer, two walks and three runs scored. On the swing through Boston and Baltimore, he went 12-for-27 with three homers and 11 RBI.
Baltimore's Rafael Palmeiro went 0-for-4, remaining eight hits short of 3,000. He has gone hitless in three straight games following a 12-game hitting streak.
Lee (9-3) allowed four runs and nine hits in 6 2/3 innings to improve to 7-1 on the road. He's 3-0 in six starts since June 1.
The Indians gave the left-hander all the support he needed in the first inning. Coco Crisp hit a one-out single, advanced on a walk to Hafner and scored on a single by Victor Martinez. Ronnie Belliard followed with an RBI single before Broussard hit a drive
over the wall in left-center, his ninth home run.
"I was just looking to hit something hard somewhere," Broussard said. "I think he got behind a few guys and had to throw the ball over the plate to get strikes. We were able to put good swings on it."
The Orioles haven't scored more than five runs since June 21, so a 5-0 deficit was almost insurmountable.
"All of a sudden, it got out of hand," Mazzilli said.
Not for Lee, who walked to the mound for the first time with a nice cushion.
"You just want to go out there and throw strikes and try not to walk anybody," he said. "I gave up a couple of solo home runs with balls down the middle, but those don't really hurt you when you have a six-run lead."
Ponson (7-6) was lifted with one out in the second after giving up an RBI single to Crisp and walking Hafner. The right-hander faced only 12 batters, yielding six runs, six hits and two walks.
"I just couldn't throw strikes. I fell behind everybody," Ponson said. "Then, when you put the ball right down the middle, you're going to get hit hard."
Ponson is 0-7 with a 9.47 ERA in 12 career games against Cleveland, the only AL team he has failed to beat.
Matos and Roberts hit solo homers in the third. It was Matos' second home run, his first since Opening Day, and Roberts' 14th -- two more than he had in his major-league career before this season.
Roberts learned Sunday that he was elected to start at second base for the AL in the July 12 All-Star Game, but his delight was tempered by the Orioles' poor play the past two weeks.
"It's exciting, but at this point we're frustrated as a team," Roberts said. "It would be nice to have that come on a little brighter note."
Cleveland made it 7-2 in the fifth against James Baldwin. Hafner and Martinez hit singles before Belliard lofted a foul sacrifice fly into the right field corner that scored Hafner without a throw.
Hafner hit his 14th homer in the seventh after Steve Kline
issued a leadoff walk to Crisp. Roberts chased Lee with a two-run double in the bottom half.
Martinez has hit in 14 straight games, tying a career high. ... Ponson's ERA went from 5.35 to 5.80. He has one win in six starts since June 1. ... Cleveland is 24-1 when scoring at least six runs. ... Cleveland's Aaron Boone went 3-for-26 (.115) on the road trip. ... Third baseman Melvin Mora returned to Baltimore's lineup after a 10-game absence with a strained hamstring and went 1-for-4.