He went 2-8-1 with the Avalanche before moving on to the Blues in the offseason where he has a 16-5-2 record.
"You really want to get back in that winning feeling and doing it here is obviously pretty big for me," Elliott said.
"It's pretty fun playing against some of your best friends over there. They're having a great year and we just wanted to come here and get a road win. We haven't been as good on the road, so coming in here for kind of a homecoming meant a lot to me and the guys played hard for me."
Anderson was making his 18th straight start, but didn't last four minutes.
Chris Porter also scored for the Blues, who won for just the second time in their past five games.
Anderson was replaced by Alex Auld after allowing two goals on just four shots. Auld turned aside 13 of 14 shots.
"The biggest thing is to make sure we're sticking together," Auld said. "That's what has gotten us through some tough times this year is the fact that we are a close group and we have to make sure that we are holding each other accountable and still having a lot of fun in practice and I believe we're going to get out of this."
The Blues entered the third period with a 3-1 lead after Perron scored his second of the game at 6:35 of the second when he chopped a backhand over the shoulder of Auld.
The goal restored a two-goal lead for the Blues after Alfredsson scored on the power play just 56 seconds into the period.
Sergei Gonchar fired a shot that hit Alfredsson's skate at the side of the St. Louis goal, bounced off Elliott and went in.
The first period was much like the second as the Senators failed to gain any big rewards for their efforts.
Despite outshooting and out chancing the Blues in the opening period, Ottawa trailed 2-0 heading into the first break.
Porter opened the scoring with a redirect just 1:37 into the game.
Perron scored his first at 3:49 after picking up a rebound that Anderson couldn't control and scoring before the Senators' goalie could get back into position.
"I don't think we were ready to play the game. You have to be ready to start the game," Ottawa coach Paul MacLean said.
"We made one execution error on the first goal obviously, and on the second goal they just skated right through us so obviously we weren't prepared to play the game."
Elliott kept Ottawa off the board with nine saves in the period, including one off Colin Greening who took a pass in tight from Jason Spezza on a 2-on-0 break.
"I think our team has come out of the break and stayed the same while the league has gone to another level," MacLean said.
"We may even have gone backward, but we certainly haven't gone to the same level as the rest of the league."
The Senators host the Nashville Predators on Thursday. It will also mark the return of Mike Fisher to Ottawa for the first time since he was traded to Nashville.
Senators defenseman Chris Phillips played his 999th game, all with Ottawa. ... Milan Michalek is halfway there as he played his 500th game Tuesday, and 183rd with Ottawa. ... The Blues are 10-0-2 against the Eastern Conference this season. ... The win was the first for St. Louis in Ottawa since Jan. 26, 2000.
Copyright by STATS LLC and The Associated Press
Arena: Canadian Tire Centre Location: Ottawa, Ontario
Referees: Justin St. Pierre, Brad Watson Linesmen: Don Henderson, Derek Nansen
Attendance: 18,952 (102.4% full)
Team Stat Comparison
Power Play Goals
Power Play Opportunities
Shots on Goal
Chris Porter (4)
Assist: Roman Polak
David Perron (5)
Assists: Patrik Berglund, Barret Jackman
Daniel Alfredsson (18)
(Power Play) Assists: Sergei Gonchar, Jason Spezza
David Perron (6)
Assists: David Backes, T.J. Oshie