The rock legend was the last of the big names to perform at the star-studded Madison Square Garden show and eased the crowd into his set with his songs Helter Skelter, Let Me Roll It, Blackbird and My Valentine, for which he was joined on piano by jazz musician Diana Krall.
He then introduced Grohl and Novoselic to the spotlight for the moment Nirvana fans had all been waiting for - the grunge stars' first performance together in almost 20 years - and they debuted a new track which had never before been performed in public.
News of McCartney's involvement with the surviving Nirvana bandmates surfaced early on Wednesday after he revealed Grohl had invited him to "jam with some mates", but the former Beatle had no idea he was stepping in for tragic frontman Kurt Cobain, who committed suicide in 1994.
However, the Nirvana reunion was short lived - after the song premiere, McCartney returned to his Beatles track I've Got A Feeling and Wings hit Live and Let Die.
But the veteran had one more surprise in store for fans and closed the show with Alicia Keys performing her chart smash, the unofficial Big Apple anthem Empire State of Mind, as he was joined by members of the New York Fire Department and first responders of superstorm Sandy.
McCartney wasn't the only performer to stage unexpected collaborations - beloved New Jersey natives Bruce Springsteen and Jon Bon Jovi got the mega show off to a rocking start with a rousing rendition of Born to Run. The rockers teamed up again shortly afterwards during Bon Jovi's set to play Who Says You Can't Go Home.
Pink Floyd singer Roger Waters duetted with Pearl Jam's Eddie Vedder on the classic Comfortably Numb, and Chris Martin made his mark by uniting with Michael Stipe for a stripped down version of his former band R.E.M.'s tune Losing My Religion.
After the rare performance, Martin quipped: "He (Stipe) came out of retirement for that and just went right back in!"
The line-up for the massive benefit gig read like a who's who of rock 'n' roll legends - Billy Joel, the Rolling Stones, Eric Clapton and The Who also hit the stage, and each act took a moment to pay tribute to the victims of the natural disaster, which devastated America's East Coast when it struck in late October.
Springsteen sent a special prayer to those hardest hit by the superstorm, while Jon Bon Jovi encouraged fans and viewers to donate to the cause, saying: "When this storm hit, we all knew that healing process would begin, but it would take a long time. This recovery is not gonna (going to) be quick, it's gonna take a while, and so we need your support, your economic support and your sweat and your hearts and your prayers because this is gonna go on but we are strong, we are New York, we are New Jersey; this is our people, this is our crowd."
Stones frontman Sir Mick Jagger added a more light-hearted tone to the event, with the Brit telling the audience: "We're so glad to be part of this event tonight. It's also wonderful to see... this has got to be the largest collection of old English musicians ever assembled in Madison Square Garden! But I just wanna say, it's great helping this, but if it rains in London, you gotta come and help us, OK?"
Actor Adam Sandler also injected a little comedy into the bash, singing a reworked version of Leonard Cohen's Hallelujah, which included the lyrics: "Hallelujah, Sandy scr** ya/We'll get through ya 'cause we're New Yorkers".
Rapper Kanye West and R&B star Keys also played solo sets as part of the line-up.
The show featured a long list of guest presenters, including Billy Crystal, Quentin Tarantino, Jamie Foxx, Katie Holmes, Kristen Stewart, Steve Buscemi, Jake Gyllenhaal, Blake Lively, Olivia Wilde and rap mogul Sean 'Diddy' Combs.
A slew of celebrities turned up to man the phones and take donations from generous viewers, such as Susan Sarandon, Ben Stiller, Whoopi Goldberg, Jeremy Piven, Martha Stewart, Gina Gershon, and the reunited cast of The Sopranos.
The 12-12-12 Concert for Sandy Relief, which ran for almost six hours, raised money for Robin Hood Relief Fund and was broadcast live on TV and online across the world to an estimated 2 billion people.