Raffa scored 14 of his team-high 22 points in the second half while Harris had 15 of his 20 in the first half.
Harris also accounted for nine of Albany's 39 rebounds.
The Great Danes (10-6, 2-1) trailed 39-34 early in the second half before unleashing an 18-5 run over a six-minute stretch to grab a 52-44 lead with 11:28 left. Vermont (9-6, 0-2) never caught up.
Albany shot 58 percent from the field and 4-of-5 on 3-pointers in the second half.