Speaking at Stormont Castle on Thursday, in the wake of the damning report into the handling of abuse allegations in the Co Cork diocese, Mr McGuinness stressed that the Executive has a duty to ensure all children are protected from abuse.
"I certainly don't want to see five years from now, or ten years from now, someone being able to turn round and say in the north of Ireland that this sort of behaviour continued because myself and Peter Robinson didn't do our job," he said.
"And I know he (Peter Robinson) is as committed to trying to ensure that we stand faithfully by the victims as I am."
Recalling the "harrowing" stories of clerical abuse victims, DUP leader and First Minister Peter Robinson added: "The kind of inquiry that we have is being considered.
"There are a couple of options we have to look at."
The comments came as the Pope's ambassador to Ireland, Papal Nuncio Archbishop Giuseppe Leanza was answering criticisms over the handling by the Catholic Church of clerical sex abuse allegations.
He had been summoned to a lunchtime meeting at Iveagh House with Tánaiste and Foreign Affairs Minister Eamon Gilmore, in the wake of a damning report into the Cloyne diocese in Co Cork.
I told him (Archbishop Leanza) the Government considered it unacceptable that Vatican intervention may have led priests to believe they could in conscience evade their responsibilities under Irish laws which could have protected innocent children from sexual abuse.
Tánaiste Eamon Gilmore
As well as blaming Newry-born retired bishop John Magee for letting down victims of abuse by priests, the inquiry - headed by Judge Yvonne Murphy - was highly critical of the Vatican.
Archbishop Leanza, who refused to take questions after the meeting, was ordered to get answers from the Vatican on the scandal.
In a brief statement outside, he said he was "very distressed" at the failures.
"I wish to stress however, the total commitment of the Holy See for its part in taking all the necessary measures to ensure the protection of children," he added.
The 400-page report branded the Vatican response as "entirely unhelpful", as it referred to mandatory rules on reporting clerical abuse as merely a "discussion document".
The Vatican's most senior spokesman, Fr Federico Lombardi, declined to answer questions on the scandal - the fourth major clerical abuse inquiry in Ireland to rock the Church.
"I think this is absolutely disgraceful that the Vatican took the view that it did in respect of something that's as sensitive and as personal with such long-lasting difficulties for persons involved," Taoiseach Enda Kenny said.
Mr Kenny has also warned that new laws, due to be rolled out in the autumn, will not be stopped by canon rules.
This is not about Ireland of long ago - it's about the Ireland of contemporary times and it's now got to be dealt with.
Taoiseach Enda Kenny
On Wednesday, Irish Justice Minister Alan Shatter insisted that priests and doctors would no longer be able to hide behind the sanctity of confession or doctor/patient confidentiality, when it came to reporting sexual abuse.
Anyone found not to have reported abuse will face a five-year jail term.
"In situations where these appalling activities took place, let them be reported and let the law of the land apply," Mr Kenny insisted.
The Cloyne report has caused dismay, as it covered a period as recent as 2009 - after new regulations were supposed to have been put in place by the Catholic Church.
But the inquiry found Rome had effectively given Bishop Magee carte blanche to ignore guidelines, as the Vatican had serious moral and canon reservations about reporting abuse.
"Of all the reports that there have been - from Ryan, Murphy, Ferns - I view the outcome of the Cloyne report as by far and away the most serious," Mr McGuinness added.
"I think it has profound implications and I certainly think that the case that has been made for investigations in all dioceses makes a very compelling argument."