Smith, who missed Wednesday's game against Seattle with a left quadriceps strain, started at center against the Bucks and scored a career-high 38 points for the Hawks.
The 17th pick in the 2004 draft, he was 13-for-22 from the field and 12 of 16 from the free- throw line. His previous career best was 32 points against Minnesota on March 10.
Smith had seven rebounds, three blocked shots and two steals and said that he was feeling 75 to 80 percent.
"I tried to come back and tried to make a stand, we fell short, but our effort is there," he said. "It was one of those games where you feel it and get it going."
Joe Johnson cooled off after scoring a season-high 39 points Friday against Seattle. He had 14 points on 6-for-16 shooting for the Hawks, who have lost four of five and are 0-4 on the road this season. Their last road win was a 96-93 victory over the Los Angeles Clippers on Feb. 14.
Bucks coach Larry Krystkowiak praised Redd's defense on Johnson.
"Mike stepped up and wanted to guard Joe Johnson tonight," Krystkowiak said.
While Milwaukee couldn't stop Smith, containing Joe Johnson really helped.
"He's a very good player, but I wanted to stop him from taking over the game," Redd said. "I worked as hard as I could to keep him in check."
Clinging to a one-point lead, Bogut tipped in a miss to put the Bucks ahead 97-94 with 2:37 left. Smith answered with a basket for the Hawks. Redd drove the lane and scored and after Johnson missed from in close, Bogut made a layup on a pass from Mo Williams, giving the Bucks a 101-96 lead with 1:14 left.
Bogut said that it was a big win, especially after Milwaukee earned a three-point win against Memphis on Wednesday. However, he wasn't completely satisfied.
"It shouldn't have even been that close," he said. "It should have been a 20-point win, but you have to give the Hawks credit for fighting back. We have to learn how to put teams away, like the Spurs did to us."
San Antonio pounded Milwaukee 113-88 on Sunday.
After Atlanta failed to score, Williams made two free throws with 33.4 seconds left and then stole an inbound pass and scored for the final margin.
Winless in four games on the road, the Bucks improved to 4-0 at home and pulled to .500 with the victory.
The Bucks couldn't distance themselves from Smith and the Hawks in the fourth quarter.
Smith, who had 15 points in the fourth quarter, pulled Atlanta to one at 95-94 with 3:03 left with two free throws after Bogut was assessed a Flagrant 1 foul on Smith's drive to the basket. Atlanta turned the ball over on the ensuing possession when Bogut took a charge from Johnson.
Milwaukee closed out the game with a 10-2 run, started by Bogut's tip-in basket and capped by Williams' two free throws and a basket off a steal on an inbounds pass.
Hawks coach Mike Woodson said the team's shot selection down the stretch hurt their chances.
"We were in the penalty and taking long shots," he said. "The clock was in our favor, but you have to penetrate the ball and try to get something at the rim, especially when you are in the penalty."
Smith had a driving layup and two free throws as Atlanta cut the deficit to 76-75 early in the fourth. Bobby Simmons hit a 3-pointer to stop the charge, but Smith's layup got the Hawks back to two.
The Bucks seemed to be in charge when they pushed the lead to 68-53 on two free throws by Yi Jianlian with 7:11 left in the third quarter before the Hawks mounted a comeback with a 17-8 run to close out the period.
Smith started the run with a rebound and dunk, a 3-point play and a free throw. Josh Childress capped it with a wide-open 3-pointer from the deep corner with 0.1 seconds left to pull Atlanta to 76-70.
"It's a great night (for Smith), but I'm happy and I'm unhappy because to me it's not about individual awards. It's about winning," Woodson said.
Smith had 12 of Atlanta's 24 first-quarter points. ... The Hawks were without guards -- rookie Acie Law (ankle) and Speedy Claxton (left knee). ... With the victory, the Bucks join the unbeaten Boston Celtics and the Detroit Pistons as the only teams with a perfect home record in the Eastern Conference. ... The Hawks lost their 15th game in the past 17 in Milwaukee. ... The Bucks' bench had 19 points in the first half while Atlanta's had eight.