On Thursday, Mr Poots faced questions from MLAs on his controversial plans which would have seen thousands of vulnerable people moved from the homes in which they live.
Last week, following emergency meetings with the heads of five health trusts, Mr Poots called a halt to the initiative.
Speaking at Stormont, he told the committee: "We failed the people who are vulnerable and elderly, and that's far and away the worst aspect of that.
"Everything else we can recover from. We can't undo the damage, even if it's for a short period of time, we can't undo the damage that was done."
On Tuesday, Mr Poots said no homes would be shut within six months, adding that he had never supported the controversial plans.
John Compton, from the Health and Social Care Board, was in charge of Transforming Your Care, which outlaid the closure of the homes.
"I want to say that I'm deeply sorry to individuals. That should never have happened. It's not what this was set out to do.
He added: "This is set out to improve the quality of people's lives, not to cause distress, not to make their lives miserable."
However, at least half of the state-owned residential homes in Northern Ireland remain earmarked for closure.
But the Health Minister said the residents will be consulted and a more sensitive process carried out.