Youngstown State had not beaten Milwaukee since Feb. 14, 2004, when it won 85-84 in Youngstown, Ohio.
The Penguins won their second straight game to improve to 4-6 overall and 1-1 in the conference.
The Panthers (2-9, 0-1) lost their fifth straight game. Milwaukee, which was 22-9 last season and went to the NCAA tournament, is off to its worst start since 1997-98 when it lost 15 of its first 16 games.
Kevin Massiah's basket ended a 16-2 Panther run with three minutes left to play for a 61-60 Milwaukee lead. But Humphrey tied the score with a free throw with 2:46 remaining and Roberts and Humphrey then had back-to-back buckets for a 65-61 Penguin lead with 1:36 remaining.
Paulsen was fouled with 1.9 seconds left and could have tied the score at 66 but he missed the second free throw. Byron Davis made a pair of free throws with 1 second left for the game's final points.
Kevin Massiah's layup with 30 seconds left gave Milwaukee a 32-30 halftime lead.
The Panthers scored 9 straight points to lead 24-19, the biggest advantage by either team in the first half. The Penguins came back to tie the score twice at 27 and 30 before Massiah's basket.