The Panthers (6-18, 3-11 Colonial Athletic Association) took their largest lead at 27-16 with 15 seconds gone in the second half on a layup by Justin Billingslea.
But the Blue Hens (11-13, 8-6) responded by outscoring Georgia State 19-7 over the next 11 minutes, claiming a 35-34 lead. Johnson and Courtney combined for 11 points during the spurt.
Georgia State went ahead 43-42 with 3:48 left in the game on a basket by Leonard Mendez, who finished with 18 points. But Edwin Santiago followed with a basket 22 seconds later to put Delaware ahead for good.
Rashad Chase scored 13 points and grabbed 14 rebounds for the Panthers.