Bryan Smolinski scored a power-play goal in the second in his Montreal debut.
Stone and Taffe beat Canadiens starter Jaroslav Halak on the Penguins' first two shots of the game to claim an early 2-0 lead.
Stone opened the scoring 32 seconds in when he put back a rebound of Jonathan Filewich's shot off the boards. Taffe stunned the sold-out Bell Centre crowd of 21,273 when his wrist shot beat Halak moments later to put Pittsburgh up 2-0.
Price drew a large cheer when he stepped onto the ice 9:45 into the second to replace Halak, who made eight saves.
Price, chosen fifth overall by Montreal in the 2005 entry draft, stopped 14 of 17 shots.
Copyright by STATS LLC and The Associated Press
Arena: Bell Centre Location: Montreal, Quebec
Ryan Stone Assist: Jonathan Filewich
Jeff Taffe Assist: Chris Minard
Bryan Smolinski (power play) Assist: Thomas Beauregard, Jamie Rivers
Petr Sykora Assist: Evgeni Malkin
Saku Koivu (shorthanded --Unassisted) Unassisted
Evgeni Malkin Assist: Alex Goligoski, Ryan Whitney