The comments from Mr Kelly - who was involved in the administration of the scheme, agreed in a deal between the last Labour government and Sinn Féin - came as the Hallett report was published.
Lady Justice Hallett led a review ordered by Prime Minister David Cameron, finding catastrophic failings - but also that the letters given to 'on the runs' did not amount to an amnesty.
"Judge Hallett has said clearly that the process was lawful," Mr Kelly said.
"The report also make it clear that the process was not secret - there are over 30 pages of material included in the report, which was in the public domain.
"This included reports in the media, questions and answers in Dáil Éireann and in the two British Houses of Parliament, Policing Board minutes and the Eames Bradley Report.
"A long list of politicians from the DUP, UUP, Alliance and SDLP discussed the issue over a period of a decade in meetings with the British government and British officials."
Mr Kelly further added: "The two governments made commitments to deal with the anomaly of the issue of the on the runs in the wake of the Early Release Scheme under the Good Friday Agreement.
We expect the governments to live up to those political commitments.
Gerry Kelly, Sinn Féin
However, DUP First Minister Peter Robinson has insisted that the scheme was "wrong in principle and shambolic in practise".
He added: "Of course, there is no amnesty and there is no get-out-of-jail-free card.
"Those are comments made by Lady Justice Hallett and in the legal definition of those terms she is, of course, right.
"But of course, it was a get-out-of-jail-free card for Downey."
The Hallett report noted: "The fact that Mr Justice Sweeney stayed the case against Mr Downey as an abuse of process does not mean that any future prosecution of another individual who was sent a similar letter would necessarily amount to an abuse of process.
"Each case will turn on its own facts."
However, Mr Robinson said that the letters could act as "get-out-of-jail-free cards" in the future.
"It has the potential of being a get-out-of-jail card for the two that Lady Justice Hallett refers to as being 'in error'. It has the potential of being a get-out-of-jail-free card for the others who were not given the caveat that was entered in some of the personal letters that were sent out," he said.
"And indeed it has the potential of being a get-out-of-jail-free card for anyone who has received a letter, because no one knows whether it would amount to an abuse of process application being accepted by a future judge."
The heart of this issue is: what happens next? What will the Secretary of State be advised to do by her legal advisers, and will she withdraw any aspect of the scheme that allows someone to rely on those letters by way of avoiding, not just prosecution, but avoiding questioning that might lead to prosecution?
Peter Robinson, DUP
Alliance Justice Minister David Ford said Thursday's Hallett report was the "first step in rebuilding confidence in the justice system".
"I am seeking assurances from the Secretary of State, the Public Prosecution Service, and the PSNI that no element of the scheme still exists," he said.
"Lady Justice Hallett has identified a further two letters sent which contained errors originating from the PSNI. The PSNI were participants in a scheme that was not properly thought through.
"A scheme initiated by ministers that evolved over time with no overall ownership or accountability and for which ministers bear responsibility. A scheme that put up barriers to prosecutions, where evidence could have been tested in open court.
"As Justice Minister I welcome the finding that the scheme did not impact on ongoing investigations. I hope the families of victims receive some comfort on this fact."
SDLP leader Alasdair McDonnell said the Hallett review showed that his party was "right at the time to warn previous Labour governments about the risks associated with side and secret deals".
He added: "Justice Hallett confirms the deep flaws in the OTR scheme as it had no structure, no policy, and no risk assessment that would allow for errors to be rectified.
"I hope that no one will use the contents of this report to impede political progress in Northern Ireland, by using it as an excuse to refuse to engage fully and honestly with each other.
"The revelations make it the re-establishment of all-party talks even more urgent."
UUP leader Mike Nesbitt said: "While Lady Justice Hallett has written a report that offers something for every political party to latch onto, it is telling that she has confirmed that Sinn Fein were the only local political party who were aware of the detail of the scheme.
"That is simply unfair. You cannot avail of a scheme if you are unaware of its existence."
He added: "There will be very little comfort for many victims in this report, who will be more concerned with assessing the morality rather than the legality of such a scheme.
"The Ulster Unionist Party awaits with interest the conclusion of the investigation being concluded by the NI Affairs Select Committee at Westminster."
William Frazer, who has campaigned for victims of the IRA, insists that there was an amnesty.
"It is now absolutely beyond doubt the British authorities gave a commitment to republicans that OTR terrorists would neither be extradited nor pursued in relation to unsolved IRA murders," he said.
"Considering the fact 98% of unsolved murders in south Armagh were committed by OTRs, this is a damning indictment of the fact political policy was put ahead of justice and human rights.