The musician had been battling the liver disease cirrhosis in recent months, and was forced to pull out of the Brown Sugar tour dates in Australia and New Zealand in October.
'The Rolling Stones are devastated by the loss of their very dear friend and legendary saxophone player, Bobby Keys,' the band said in a statement. 'Bobby made a unique musical contribution to the band since the 1960s. He will be greatly missed.'
Michael Webb, who played keyboard with Keys, said Keys died on Tuesday after a lengthy illness. Keys had been out on tour with the Stones earlier this year before his health prevented him from performing.
The Texas native began his music career as a teenager, touring with Buddy Holly and Bobby Vee when still in his teens.
He befriended the Rolling Stones in 1964 and was later recruited to play on their 1969 album Let It Bleed.
Keys was a mainstay on the British band's albums until 1974. He reunited with Mick Jagger and Keith Richards and pals in 1980 for their Emotional Rescue project.
He remained a key player on all subsequent albums, including 2005's A Bigger Bang.
Keys' biggest contributions were on hits such as Brown Sugar, Can't You Hear Me Knocking and Happy.
He also played with Eric Clapton, Lynyrd Skynyrd and late Beatles stars John Lennon and George Harrison.