Rivers' team was coming off an impressive road win at Miami heading into Saturday's game at the Grizzlies, who trailed by as much as 11 points before tying it up late in the fourth quarter.
"We didn't close out the game," said Rivers, whose team went 3-1 on the road trip. "We had great shots offensively. But I was just upset we let them score twice back-to-back. We'll take the trip."
Boston had a four point lead late in the fourth quarter, but Rudy Gay's short pull-up jump shot for Memphis with 7.3 seconds left in regulation tied it at 99.
Celtics forward Glen Davis hit a jump shot and Allen dropped in an arching 3-pointer to give the Celtics a 104-99 advantage in overtime.
Pierce made four free throws in the final 1:30 to seal Boston's seventh win in its last eight games.
Gay had 22 points for the Grizzlies. Memphis forward Zach Randolph scored 21 points and grabbed 11 rebounds.
Celtics center Shaquille O'Neal scored 18 points in his second game back after missing five with a bruised knee, and Rivers called his play "phenomenal." Rajon Rondo had 11 points and 17 assists, the fourth time he has reached or exceeded that total this season.
"Excellent trip," said O'Neal, whose 18 points were his season high. "We're 8-2, and if we can go 8-2 out of every 10, I will be very happy ... We should be 10-0, so we're playing well."
Allen scored 15 points after going for a season high 35 in a 112-107 win at Miami on Thursday. Allen scored 13 of his points in the fourth quarter and overtime.
"Guys set great picks for him," Rivers said. "Ray doesn't need but one. Once one goes in, it usually means several are going to follow."
Memphis grabbed its first lead of the game when a short shot by Gay made it 86-85 with 6:46 left in the game.
A layup by Gay gave the Grizzlies a three point lead, but the Celtics went on an 8-2 run to take a 97-94 lead on a Davis layup with 3 minutes left.
Allen's jumper made it 99-95 before Gay scored two baskets in the final 39 seconds to force overtime.
"We let them get back into it," O'Neal said. "Last game on the road trip, and that's not really an excuse, but we let them get back into it. But we had five extra minutes to calm down, and I think we did. Guys hit some big shots."
Boston is 3-0 in overtime this season, while Memphis is 0-2. The Celtics now have five consecutive wins at FedExForum.
Grizzlies center Marc Gasol praised his team's second half rally. Memphis tightened up its defense to hold Boston to just three points over the final 4:07 of the third quarter. Two free throws by Gasol tied it at 73, igniting the crowd heading into the fourth.
"That's how good teams play," said Gasol, who had 14 points and six rebounds. "As soon as the fourth quarter started, things started getting hotter and every basket was a dogfight."
Grizzlies owner Michael Heisley honored Gay before the game with a Rolex watch for his contribution to the U.S. men's national team's gold medal performance at the 2010 FIBA World Championships. Gay averaged 7 points and 2.9 rebounds in nine games in Turkey.
Boston's roster has 109 seasons of combined NBA experience. Memphis' has 36. ... Celtics center Jermaine O'Neal (sore left knee) was not with the team ... Grizzles guard Mike Conley and Rondo are two of only three NBA players this season with more than 60 assists and 20 steals. Washington rookie guard John Wall is the third ... The Grizzlies' last home win against the Celtics was a 93-76 victory on March 14, 2006.