Senior judges declared significant unease about the safety of the verdict returned against Patrick Livingstone for the so-called 'Good Samaritan' killing of Samuel Llewellyn.
Their ruling was based on the alleged brutality of RUC officers involved in securing a statement implicating the west Belfast man.
Mr Livingstone, 62, described the decision was a vindication of his fight to clear his name.
He now plans to seek compensation for the 17 years he spent behind bars for the murder.
Mr Llewellyn was abducted as he delivered hardboard to repair homes damaged by a bombing on the city's Falls Road in August 1975.
The council cleansing department worker was taken to a house in Leeson Street where he was shot dead.
His body was wrapped in a sheet and put in the back of a van which was then set alight.
The only evidence against Mr Livingstone at his trial came from three RUC officers who later interviewed him at Dundalk Garda Station and claimed he confessed to the murder.
It was alleged that he taunted the policemen about the shooting, boasting they could do nothing about it because he had no intention of crossing the Irish border.
He disputed their account and denied the killing, with his defence claiming the RUC witnesses had concocted a lying account.
However, he was subsequently convicted at Belfast City Commission in May 1977 and sentenced to life imprisonment.
I feel totally vindicated. But there's a lot more people than me, on both sides of the divide, who went through those Diplock courts.
His case was reopened and referred back to the Court of Appeal by the Criminal Cases Review Commission (CCRC), the body set up to examine potential miscarriages of justice.
The challenge centred on alleged police brutality towards another man who said he was beaten into signing a statement which claimed Mr Livingstone admitted the shooting to him.
His allegations included: being put against a wall and hit across the stomach; having chest and head hairs pulled out; and being hooded, spun around and hit across the feet for up to 45 minutes.
The CCRC also investigated the quashing on appeal of another man's convictions for assaulting two of the RUC officers who testified at the murder trial.
Ruling on the case alongside two other judges, Lord Chief Justice Sir Declan Morgan held that the fresh evidence should be introduced.
He said the claims of police mistreatment was never tested and would have opened a line of inquiry "which might have affected the credibility of the police witnesses".
Sir Declan added: "Because of the non-disclosure the appellant lost the opportunity to pursue that line of argument."
Evidence had also been raised of potential wrongdoing in testimony from at least some of the police interviewers, the judge held.
He confirmed: "For the reasons set out we have a significant sense of unease about the correctness of this verdict and accordingly allow the appeal."
Mr Livingstone, who was in the Court of Appeal with his son Cormac and other friends, told how the outcome has been "a long time coming".
Speaking after the verdict, he said he felt "totally vindicated".
He said: "I spent seventeen and a half years in jail for something I didn't do. I intend to seek compensation."