Ugoh took over the position when Tarik Glenn retired after the Colts won the Super Bowl in 2007. Since then, Ugoh has started 23 regular-season games, helping protect quarterback Peyton Manning's blindside.
He'll have to work a bit harder to regain his spot on the depth chart.
"I always look forward to a challenge," the soft-spoken Ugoh said Tuesday after a morning practice. "I don't like getting beat. I'm a perfectionist. I take getting beat kind of hard."
Caldwell said he told Ugoh of the decision before organized team activities, and Ugoh didn't improve enough during the offseason to change Caldwell's mind.
"He's a talented guy," he said. "His rookie year, I think he performed well. We're just anticipating that he's going to make a leap here shortly. This is his third year, and we're just looking for overall improvement."
Overall improvement is needed on the entire offensive line. The Colts were 31st in the NFL with 79.6 yards rushing per game last season.
"Where we ranked in rushing, that's just not going to get it done," Colts owner Jim Irsay said. "We have to prove we can run the ball. The negative yards lost, we can't let that happen, either.
"You can't have a glaring weakness on a championship team. Those things will catch up to you. I thought we weren't running the ball well enough last year, and that came back to haunt us," he said.
Ugoh practiced with the first team at right tackle Tuesday afternoon because Ryan Diem was held out with an undisclosed injury.
Ugoh battled through a groin injury last season. He missed one game and was a reserve the next three games before regaining his starting position for the rest of the season.
"I was dealing with injuries last year, and since then I haven't really been able to do too much," he said. "I felt as if I handled myself well, but the only thing I can do is continue to work."
The move isn't entirely a demotion for Ugoh. Johnson started 12 games at right tackle and four games at left tackle last season. He also replaced Diem in the second half of the Super Bowl in 2007 and played so well that Manning said he didn't notice immediately Diem was on the sideline.
Caldwell said Johnson is versatile and knows the system well.
"He's been able to hold down that tackle position pretty solidly when he's been in there," Caldwell said.
Johnson said his versatility is a product of having to fit wherever he could the past three years.
"That's what a lot of these young guys do, they play everywhere, and that's what I did," he said. "You just try and get as many reps as you can in different spots because that's how you're going to make the team. ... I go play wherever. I don't say much and I just go out and do it."
Ugoh said he will practice some at right tackle for the first time, where he'll back up Diem.
"The only thing I can control is what kind of effort I give and how I handle the situation," he said.
Ugoh said he won't put any extra pressure on himself Friday.
"Nothing changes for me at all," he said. "My mindset is to get better each and every day. It was my mindset before. I was never lackadaisical or comfortable in the position I was in, I was always striving to get better."