Mark McAndrew paced Brown (6-14, 1-4 Ivy) with 15 points and five rebounds.
Zoller, the Ivy League's leading scorer, came into the game averaging 18.9 points per game average. He had 16 points in the first half on 6 of 9 from the floor, and made all three of his 3-point attempts.
The Quakers (12-7, 3-0) scored on 7 of 10 3-point attempts in the first half, and 12 of 20 in the game while Brown struggled, making one of 11 3-pointers in the first half.
Penn led by as much as 18 in the first half, and held a 41-26 advantage at intermission.
The contest marked the return of former Brown head coach Glenn Miller, who coached the Bears from 1999-2006 before taking the position with the Quakers earlier this season.