"It's a pretty good feeling," Kubel said. "It's an even better feeling that we finally beat them."
Rivera replaced Joba Chamberlain (1-2) with the bases loaded and a 3-1 lead and walked Jim Thome. Kubel then pulled a 1-0 pitch into the seats in right, startling the crowd of 46,628 at Yankee Stadium.
"You probably should have recorded that. Take pictures. It's probably not something you will see often," Twins manager Ron Gardenhire said of Rivera's off day. "He's the best in the game."
Kubel's third homer of the season and sixth career slam snapped a string of 51 straight converted save opportunities for Rivera at home, tying the major league record set by Eric Gagne with the Dodgers. It was the first grand slam off Rivera since Bill Selby's game-ending shot for Cleveland on July 14, 2002.
"You assume he's going to close every single game," Yankees slugger Mark Teixeira said. "If you ask people in New York, they probably think his save percentage is a thousand, because that's how good he is."
Rivera hadn't allowed a run in 11 innings this year. The ace closer issued his fourth career bases-loaded walk and first since Oakland's Keith Ginter on May 6, 2005, before allowing his fourth grand slam, according to STATS LLC.
"Home run, anything can happen. But walking a run in is unacceptable," Rivera said. "You can not go in there and do that."
The Yankees tried to rally in the ninth, starting the inning with a pair of singles off Jon Rauch. But the closer struck out Derek Jeter, Brett Gardner and Teixeira to hold on for his 10th save in 11 chances.
Nick Blackburn (4-1) allowed three runs and nine hits in seven innings for Minnesota, which picked up its first win against New York since Aug. 13, 2008. The Twins had lost 12 straight overall against New York, counting the sweep in last year's AL division series.
"This is as good as it has been," said Blackburn, who has won three consecutive starts. "There was no point where I had to make adjustments. The last couple of games I was making adjustments during the game."
Randy Winn had a two-run triple and Teixeira collected two more hits for the Yankees, who had won six straight at home. Teixeira is batting .339 (21 for 62) with 20 RBIs in 15 May games after hitting .136 in April.
The Yankees wasted an effective start by Sergio Mitre while falling to 12-3 at home heading into a pair of difficult two-game sets against Boston and AL East-leading Tampa Bay. Phil Hughes is scheduled to face Red Sox right-hander Daisuke Matsuzaka on Monday night.
"I just look at one [series]. I'm not going to look ahead," manager Joe Girardi said. "Obviously, you want to continue to try to win series."
Gardenhire lamented all of Minnesota's missed opportunities against the Yankees before the game, and he got a couple more to add to his list in this one. But he also got just enough for Minnesota's first victory in New York since July 4, 2007.
"As far as I'm concerned, we're 1-0 against the Yankees in our last one game played," he said. "We'll construe the numbers any way we want now."
Mitre allowed one run -- Justin Morneau's ninth homer -- and four hits in five-plus innings. He could start Friday night's Subway Series opener against the Mets if Girardi decides to use Javier Vazquez in long relief over the next couple days.
"Felt pretty good. Everything was pretty much working," Mitre said. "Throwing a lot of strikes, trying to stay ahead of the hitters and just pound the strike zone throughout the whole game."
Yankees CF Curtis Granderson took batting practice and fielded about 15 balls in the outfield for the first time since he strained his left groin May 1. He also ran again and did some stretching. "Everything felt good," he said. ... Twins 3B Nick Punto was out of the starting lineup because Gardenhire wanted to give Matt Tolbert playing time. Tolbert went 1 for 3 in his first game since he was recalled from Triple-A Rochester last Tuesday. ... Girardi said RHP Chan Ho Park (right hamstring) likely will come off the DL on Monday. ... The Twins open a two-game series at Toronto on Monday. RHP Kevin Slowey is scheduled to face Blue Jays LHP Dana Eveland.