Outman (1-0) earned his first major league victory since June 8, 2009 against the Minnesota Twins.
Outman and Devine missed last year following Tommy John surgery.
"Knowing what we both went through to get back to this point and taking part in a team win feels nice," Outman said. "My first win in a while. That's a plus."
"This has always been, this time of year, an unfriendly place," Orioles manager Buck Showalter said. "But we had some opportunities and we just couldn't take advantage of it."
Outman, in his second start since returning to the majors for the first time in two years, allowed two runs and six hits over six innings. He walked five.
"I definitely pitched myself into some jams," Outman said. "Walks -- I need to cut back on those."
Brad Bergesen (1-6) allowed three runs and nine hits in 5 2/3 innings. He walked two and struck out five.
"It doesn't matter who pitches the eighth or ninth," Fuentes said. "Andrew (Bailey) is a good closer and he's working his way back. When he gets here he's probably the best man for the job."
Fuentes recorded his first save in his first save opportunity since May 8.
Crisp, Ellis, Kouzmanoff and Cliff Pennington each had two hits for the A's, who recorded double-digits in hits for the third consecutive game for just the second time this season.
"I felt like I made my pitch to Pennington and he was just able to get enough of it and put it in play," Bergesen said. "That's a tough one. There were times I felt really good."
Ellis ended a 50-game homerless streak with his solo shot in the eighth, making it 4-2.
Crisp put the A's ahead with a two-out RBI single in the sixth.
The A's scored two of their runs with two outs and have scored seven of their last 10 runs with two outs.
The Orioles did not get a runner past first base since they left the bases loaded in the fourth.
O's IF Mark Reynolds started at first base for the first time in two years. ... Devine has not allowed an earned run in his last 31 games (31 1/3 innings) dating to 2008. ... O's Brian Matusz (left intercostal muscle strain) is set to throw a bullpen session Sunday. ... O's IF Derrick Lee (left oblique strain) is expected to start baseball activities at extended spring training Sunday. ... Pennington has hit safely in a career-high matching seven games.