The All-Star left-hander allowed one hit in six scoreless innings, and the Rays' bullpen weathered ninth-inning home runs by Derek Jeter and Alex Rodriguez to beat the Yankees 7-5 and avoid a three-game sweep on Thursday night.
Willy Aybar homered and drove in three runs for the AL East leaders, who avoided a three-game sweep and extended their lead in the division over second-place Boston to 3 1/2 games. The Red Sox, who face Tampa Bay six times over the next two weeks, were idle.
The Yankees' slim playoff hopes were dealt another blow. They trail the Rays by 11 games in the division and Boston by 7 1/2 in the wild-card race.
"It's good to keep them in our rear view mirror because they've played well against us," Tampa Bay manager Joe Maddon said. "To lose three in a row here would not have been any fun at all."
Kazmir (11-6) limited the Yankees to Cody Ransom's fifth inning double, walked five and struck out seven, but a high pitch-count (98) prevented him from going deeper into the game. Ransom also singled off reliever Trever Miller in the seventh for New York's second hit.
Chad Bradford worked a scoreless eighth but Tampa Bay lost its bid for its 13th shutout of the season in the ninth when Jason Hammel gave up RBI single to Ransom, who finished 3-for-3, and home runs to Jeter and Rodriguez.
Jeter's three-run shot to right field moved him within four hits of tying Babe Ruth for second on the Yankees career hit list. Rodriguez followed with career homer No. 550 to trim New York's deficit to 7-5.
"We can't look [at not gaining ground on the Red Sox]. ... How many games do we have left? 22. We have to go out and do what we do," Rodriguez said.
"We're not looking at series. We're looking at games as must-win games, and that's the situation tomorrow. It's almost like college. Double-elimination and you go home. That has to be the approach. Urgent."
A-Rod homered for the third straight night, giving him 32 this season. He had 422-foot blast upheld in baseball's first use of instant replay on Wednesday night, but there was never a doubt Thursday night's 436-footer that also struck one of the catwalks that support the roof at Tropicana Field.
"I think it might have been the farthest one I've seen here. I truly do," Maddon said. "That was zero gravity right there."
Aybar capped a five-run second inning against Darrell Rasner (5-10) with a two-run single for Tampa Bay. He hit a solo homer in the fourth off Alfredo Aceves.
The Rays, who lost a series for the first time since the All-Star break, looked bad in dropping the first two games.
Maddon attributed the flat performances to the way the Yankees have been playing lately and uncharacteristic mistakes by the Rays -- not the pressure of being in a pennant race in September for the first time.
Tampa Bay improved the best record in the majors to 85-52, but is 6-9 against New York, one of two AL teams with winning marks against the Rays this year.
Kazmir walked four in the first three innings, including A-Rod twice. He didn't give up a hit until one out in the fifth when Ransom doubled on a fly ball to shallow right that fell in when Gabe Gross tried to make a shoestring catch.
"We gave them that 5-spot, and Kazmir has been tough on us all year. He shut us down for six innings, and it makes it difficult," Yankees manager Joe Girardi said. "We had a few opportunities, not a lot, and we weren't able to capitalize."
Rasner, 0-3 in nine starts since last winning on July 12, allowed five runs and six hits in 1 1/3 innings. Aybar drove in the last two runs charged to the right-hander with a bases-loaded single off Aceves.
"We had a chance to sweep them. That would have been big for us, but we didn't. You've got to move on," Girardi said.
"We need more than series. Obviously we need to string together a lot of wins. We've got a lot of ground to make up."
Since taking over for injured rookie All-Star third baseman Evan Longoria, Aybar has five homers and 16 RBI's in 23 games. ... The Yankees plan to recall OF Melky Cabrera from Scranton/Wilkes-Barre on Friday. ... The Rays improved to 52-21 at home. ... Yankees RF Bobby Abreu was scratched because of a jammed left wrist, suffered Wednesday night when he slid into second base on a ninth-inning stolen base. X-rays were negative and he is day-to-day. ... New York also rested Hideki Matsui, who had been the designated hitter in 15 straight games after being activated from the 15-day disabled list on Aug. 19. He pinch-hit for Brett Gardner in the ninth and struck out.