"We take a lot of pride because we're doing it the right way. We're doing it defensively," Williams said. "We're executing in the second half. Now we've got to find a way to do it in the first half. We start off sluggish, so maybe we need to shake things up earlier."
The Hawks have lost four straight, including three at home, after winning their first six.
Atlanta led 74-63 with 36 seconds gone in the fourth period before the Jazz put together a 15-4 run to tie it at 78 with 6:43 left on a 3-point basket by Andrei Kirilenko.
"We just hung in there when we got down a little bit," Williams said. "Just like the last games, we hung in and didn't hang our heads."
The Jazz tied it at 86 on a jumper by Williams with 1:56 left. Millsap's basket put Utah up 88-86 and the Hawks still had a chance when Josh Smith rebounded Millsap's miss with 14.7 seconds remaining, but Al Horford, fouled with 7.3 seconds left, missed both shots and Utah rebounded.
"I can take the blame for not being able to knock down those free throws at the end. I just have to be ready the next time to knock them down," Horford said.
The Hawks made only 11 of 21 free throw attempts.
"We got hurt at the free throw line," coach Larry Drew said. "The combination of not making free throws and giving up 15 offensive rebounds to a good team is not going to get you a win."
Williams, who led the Jazz with 24 points and 10 assists, was fouled with 6.4 seconds left and converted both to wrap up the game.
Al Jefferson added 15 points and 10 rebounds, and Kirilenko had 14 points and 12 boards. Millsap finished with 11 points.
The Hawks were leading 86-81 on a pair of free throws by Smith with 3:34 left. Utah, however, got a three-point play from Millsap with 3:19 and then the jumper by Williams to tie it.
Joe Johnson led the Hawks with 23 points. Smith added 20 points and 13 rebounds.
In Utah's previous three games, the Jazz beat the Los Angeles Clippers 109-107 in double-overtime at home after trailing by 16 points at the half.
The Jazz then went on the road and overcame double-digit deficits on Tuesday and Wednesday at Miami and Orlando. After a 22-point comeback to win 116-114 in overtime at Miami, the Jazz fell behind by 18 in the third quarter against Orlando before winning 104-94.
The Hawks led 45-42 at the half, with Johnson scoring Atlanta's final eight points and 14 total in the half.
Atlanta F Marvin Williams, who missed four games with a right knee injury, started. Teammate Maurice Evans, who hasn't played since the season opener due to a sore right knee, also played. ... Deron Williams and Smith each had 11 points in the third period.