Swisher has been hampered by a sore left biceps and manager Joe Girardi wants the switch hitter to wait another day before he tries batting from the left side.
Right-hander Nick Blackburn was scheduled to start Sunday's game for the Twins.
The outfielder went 1 for 4 and scored a run in New York's 7-1 victory over Minnesota on Saturday.
Swisher says he feels much better and is available to pinch-hit against a left-hander.
Marcus Thames got the start in right and was batting seventh.