Mr Reynolds had been battling Alzheimer's disease.
He was born in Rooskey, County Roscommon in 1932 and was first elected to the Dáil for the constituency of Longford/Westmeath in 1977.
He held a number of ministerial roles including finance, industry and transport.
First Minister Peter Robinson and deputy First Minister Martin McGuinness on Thursday afternoon paid tribute to the former Taoiseach and passed their condolences to his family.
DUP leader Mr Robinson said: "While we were from different political traditions and did not share the same political outlook, Albert Reynolds undoubtedly contributed to helping find an exclusively peaceful way forward in Northern Ireland.
I offer my sincere condolences to his family, friends and many colleagues in the Irish Republic
"I extend my sympathy to the family of Albert Reynolds on the occasion of his passing."
Mr McGuinness of Sinn Féin said: "I was deeply saddened to hear of the passing of former Taoiseach Albert Reynolds. I want to offer my sympathies to his wife Kathleen and family.
"Albert was a key player in bringing peace to this country. He recognised that dialogue and negotiation was the key to opening the door to peace, and had the courage and tenacity to see it through. In doing so he was instrumental to building a better future for us all."
Mr Reynolds succeeded Charles Haughey as the leader of the Fianna Fáil Party in 1992 and oversaw two political coalitions as Taoiseach until 1994.
One of the late Taoiseach's main achievements was the progress he made in advancing the Northern Ireland peace process.
On 15 December 1993, Mr Reynolds signed the Joint Downing Street Declaration in London with the British Prime Minister of the time John Major.
In a heartfelt message, the former Prime Minister described him as a friend and a politician deserving of his place in history.
Albert Reynolds was at the heart of the success of the Irish peace process.
Former Prime Minister John Major
"Without Albert, it may never have started - or might have stalled at an early stage - and Ireland, North and South, might still be enduring the violence that scarred daily lives for so long," Mr Major said.
"Albert cared about achieving peace and took risks to deliver a future for Ireland that many thought was impossible. He deserves an honoured place in the history of his country.
"To me, he became a friend I cherish and will miss."
Mr Reynolds was also instrumental in the securing of the IRA ceasefire in 1994 - a role for which he was nominated for a Nobel Peace Prize.
He stood down as Taoiseach and party leader on 16 November 1994 but stayed on as a TD in Dáil Éireann until 2002.
He is survived by his wife Kathleen and seven children.
Sinn Féin president Gerry Adams has expressed his condolences to the family of the former Taoiseach.
"I'm really sorry to hear of the death of Albert Reynolds. Albert acted on the North when it mattered," he said.
"My thoughts are with Kathleen and all the Reynolds family. May he rest in peace."
Ulster Unionist party leader Mike Nesbitt also passed his condolences to Mr Reynold's family.
"To an extent Albert Reynolds and John Major are the forgotten men of the peace process," he said.
"Certainly, in replacing Charles J Haughey as Taoiseach, Albert Reynolds brought a new dynamic to Anglo-Irish relations, and made things possible, including the 1993 Downing Street Declaration.
"It is a time for Unionists to pass their condolences to the Reynolds family, and stand back to allow the Republic to mourn the passing of a Prime Minister."
He demonstrated integrity, determination and great courage in his pursuit of peace at a time when it was so necessary.
Former SDLP leader John Hume
Current SDLP leader Alasdair McDonnell said that the former Irish leader will be remembered "as a man dedicated to peace in the face of those continuing to use violence".
While Northern Ireland Secretary of State Theresa Villiers said: "I heard with great sadness of the death of Albert Reynolds.
"He will be long remembered for his courageous and imaginative contributions to the peace process.
"His work, in close partnership with John Major and the British Government, was critical to its success. We all owe him deep gratitude for the role he played."
Current Taoiseach Enda Kenny paid a glowing tribute to the former leader's time in office.
"Albert Reynolds brought an energy and drive to the development of business and economic growth during his tenure in office as Minister for Industry and as Minister for Finance," he said.
"As Taoiseach, he played an important part in bringing together differing strands of political opinion in Northern Ireland and as a consequence made an important contribution to the development of the peace process which eventually led to the Good Friday Agreement."
Also paying tribute was Irish President Michael D Higgins, who said: "I have learned with great sadness of the death of former Taoiseach and a member of the Council of State, Albert Reynolds. I wish to express my deepest sympathy to Kathleen and the entire Reynolds family.
"Albert Reynolds will be remembered as a most dynamic Cabinet Minister and a Taoiseach with courage, who made a very important contribution to the dialogue which led to the Northern Ireland peace process."
Taoiseach Albert Reynolds with Gerry Adams and John Hume pictured during a historic meeting in Dublin in 1994.