Pittsburgh's Dany Sabourin made 21 saves.
Bryan Smolinski scored a power-play goal in the second in his Montreal debut.
Stone and Taffe beat Canadiens starter Jaroslav Halak on the Penguins' first two shots of the game to claim an early 2-0 lead.
Stone opened the scoring 32 seconds in when he put back a rebound of Jonathan Filewich's shot off the boards. Taffe stunned the sold-out Bell Centre crowd of 21,273 when his wrist shot beat Halak moments later to put Pittsburgh up 2-0.
Price drew a large cheer when he stepped onto the ice 9:45 into the second to replace Halak, who made eight saves.
Price, chosen fifth overall by Montreal in the 2005 entry draft, stopped 14 of 17 shots.