"Jered did what you want a guy to do, continue to the pound the zone," Angels manager Mike Scioscia said. "Make your pitches."
The defending AL East champion Rays fell to 0-4. Manny Ramirez was hitless in four at-bats with three strikeouts and is 1-for-16 overall, while Johnny Damon returned after missing a game with a sore right calf and singled, making him 1-for-11 this year.
"We didn't expect to start that way," Damon said.
Weaver (2-0), the major league strikeout leader last season, allowed one run and three hits in 6 2/3 innings. It is the first time since his rookie season in 2006 that he has started with consecutive victories.
Walden earned his first save with a perfect ninth that included a strikeout. He replaced Fernando Rodney, who was dropped from the closer role to work on his mechanics.
"Being a reliever, you always want to have the ball in your hands at the very end of the game," Walden said. "When I found out, I was very excited."
The 23-year-old Walden had one save last year, when he made 16 relief appearances in his first big league season.
"It's good to nail those down," Scioscia said. "Jordan has a clean ninth. I think he's confident. The talent's there. Hopefully he'll take off and run with it."
Los Angeles' Bobby Abreu reached base for the eighth consecutive plate appearance with a first-inning single. He went 5-for-5 with two walks in Sunday's 12-9, 13-inning loss at Kansas City. Abreu's streak ended when he flied out to deep center in the second.
Hunter had an RBI single and Callaspo drove in a pair with a single as the Angels went up 3-0 in the first. Hunter entered 0-for-7 with runners in scoring position.
"Anytime you get four or five runs to work with, it makes your job a lot easier," Weaver said.
"It's really hard for me to get worked up," Tampa Bay manager Joe Maddon said. "I like the work ethic. We're going to be fine."
Rodriguez pulled the Rays within 4-1 on his third-inning solo homer. Hunter extended the Angels' advantage with an RBI single in the fifth.
The Rays loaded the bases with two outs in the seventh, but failed to score when Damon lined out to left against reliever Hisanori Takahashi.
Upton hit a two-run homer in the eighth that cut the deficit to 5-3.
Upton is the only Tampa Bay player with a hit in all four games this season. ... Los Angeles INF Erick Aybar didn't play because of tightness in his left side. ... Angels 1B Kendrys Morales (left ankle surgery) could increase his running program this weekend. ... Los Angeles 2B Howie Kendrick went 2-for-2 with two walks. He is 43-for-97 in his career against Tampa Bay.