Vettel's Red Bull was .002 seconds faster than Alonso's Ferrari in qualifying to give the German his sixth pole of the season -- and his third straight.
"We knew that it would be close, but did not expect it to be that close. It's not very comfortable," said Vettel, who now has 11 poles in his career.
Vettel will be looking for his third win of the season.
Alonso won the opening race of the season in Bahrain but Ferrari has since fallen behind Red Bull and McLaren.
"Finally we got close to the Red Bulls, after 10 races," Alonso said. "We are in the front row for the first time and we lost the pole by a very small margin, but that's not very important."
Ferrari has had a good weekend so far, with Alonso faster than Vettel in Friday's practice and just behind the German in the third practice session before qualifying.
Vettel's fastest lap on the 2.8-mile Hockenheimring circuit was 1 minute, 13.791 seconds. Alonso clocked 1:13.703 and Massa 1:14.290.
Vettel said the Ferrari cars have been "very competitive" in the first two days.
"We have a very strong car, but we are very close to the Ferraris," he said. "We start in the best position and we'll go to sleep with a good feeling but we'll have to confirm it tomorrow.
"If you go over the limit you lose some grip, the tires slide and that's what happened to me in the last round today. It was enough to stay ahead, really by nothing. It was very tough mentally; I had to push very hard."
Webber, coming off his victory at the British Grand Prix two weeks ago, made a mistake at the start of his final lap and abandoned the last chance to improve his time.
Webber's victory in Silverstone was marred by a team dispute that both drivers now insist is behind them.
Webber was forced to hand over a key car part to Vettel before the race, prompting an angry remark by Webber after he won the race in Silverstone.
Michael Schumacher, who won four times in Hockenheim, qualified in 11th place in his homecoming. The seven-time Formula One champion is in his comeback season after coming out of retirement.
Schumacher was knocked out of the top 10 in the final moments of the second qualifying session by the Williams of Nico Hulkenberg, one of six German drivers in the 24-man field.
The first qualifying session was interrupted after the crash of the Force India driven by Tonio Liuzzi. The Italian driver was not hurt. He went wide on a wet patch exiting the last corner and crashed into a wall. The track later dried out.
Hamilton leads the standings with 145 points, followed by Button with 133, Webber with 128 and Vettel with 121. Alonso is fifth at 98.
Sunday's German GP is the 11th of the season's 19 races.