Pastor James McConnell sparked controversy when he likened "cells" of Muslims in Britain to the IRA and told his congregation that Islam was a "heathen" doctrine which had been "spawned in hell".
The preacher, from the Whitewell Metropolitan Tabernacle on the Shore Road in north Belfast, has already told UTV that he stands by his comments and has rebuffed calls for an apology.
Representatives of the Muslim community in Northern Ireland have expressed shock.
But, in an interview with the Irish News, DUP leader Peter Robinson said it was "the duty of any Christian preacher to denounce false doctrine".
The politician, who attends the church in question, added: "He's perfectly entitled to do that - it's an appropriate thing for a minister to do."
Mr Robinson said he did not trust Muslims who were involved in "terrorist activities" or those "fully devoted to Sharia law".
According to the First Minister, not trusting someone does not amount to hate crime, using an example of the media not trusting politicians.
This was a preacher speaking to his congregation ... The remarks were never intended to suggest hatred to any community.
First Minister Peter Robinson
In a later interview with UTV, Mr Robinson said he stood by the comments and that his interview had been taken out of context.
He said: "Of course there are people out there who are up to mischief and want to sensationalise what was said.
"The reality of course is that many Muslims in Northern Ireland have made a very significant contribution to life in Northern Ireland.
"But would I trust them for spiritual advice? No, I wouldn't and that's a matter of fact."
When asked about those Muslims who have said they feel aggrieved by the First Minister's comments, Peter Robinson added: "The people who will be offended would be those who are Muslim terrorists, those who carry out Sharia law, and those who want to offer me spiritual advice. Those are the only people who I referred to.
"I respect people who carry out their own worship and faith, I may not have respect for their religion. And indeed as a Christian, I am not going to support any other religion.
Mr Robinson also said: "Sometimes elements of the media take something embellish it and inflate it until it becomes a major issue. It's the media that have blown this issue up."
I can understand people who, when they hear some of the comments publicly which have not been representative of what I said, might have got the wrong end of the argument presented.
First Minister Peter Robinson
Reacting on Wednesday, Dr Raied Al-Wazzan from the Belfast Islamic Centre invited Mr Robinson to visit him.
"We have got lots support from the major politicians in Northern Ireland, we have lots of support from churches in Northern Ireland and lots of ordinary people and organisations phoned us and emailed in support of us," he said.
"So I would say Mr Robinson does not represent the vast majority of people in Northern Ireland."
The comments by Pastor McConnell have been condemned by Sinn Féin deputy First Minister Martin McGuinness, who also branded the speech as "hate-mongering".
"Coming in the wake of recent spate of disgraceful racist attacks against families in parts of Belfast and elsewhere, such inflammatory comments only serve to fuel hatred." he said.
Mr McGuinness has since added that Mr Robinson needed to show leadership on the issue.
All of us in positions of leadership have a responsibility to represent and stand up for all the people of our society.
Deputy First Minister Martin McGuinness
"We have a duty to promote equality, mutual respect and tolerance for all in our society based on the core principles contained in the Good Friday Agreement," the deputy First Minister said.
"I value the diversity and multicultural nature of our society the significant and valuable contribution the Muslim community makes to this society day and daily.
"There is a real need for all of us those in positions of responsibility to step out of our own political constituencies and religious groupings and show genuine political leadership for all."
However, his partner in government hit back and told UTV it "jarred to be lectured on the need for respect and equality" from a former leader of the Provisional IRA.
Despite the clash of words, the First and deputy First Minister were later pictured together in Stormont to meet the Turkish Ambassador Ünal Çeviköz.
Those in a position to influence others should not use language which may increase the vulnerability and legitimate fears of people who already feel isolated and under attack because of their race and religion.
Equality Commissioner Dr Michael Wardlow
Secretary of State Theresa Villiers said Pastor McConnell's remarks were "unacceptable" and could not be justified.
She said she condemned the remarks because of the risk they presented for individual members of the faith.
The cabinet minister refused to be drawn on Peter Robinson's remarks, because she had not seen them, but said it would be "likely"" that she and the First Minister would discuss the issue.
"From what I gather, his remarks were not without qualification," she told UTV.
The Equality Commission for Northern Ireland has called Pastor McConnell's remarks on Islam "unacceptable" and expressed concern over the subsequent intervention by the First Minister.
Chief Commissioner Dr Michael Wardlow said: "It is incumbent on all leaders in public life to demonstrate in what they say, and what they do not say, a real and true respect for all human beings.
"We have seen an alarming increase in race attacks across our community over a relatively short period. Everyone who lives in Northern Ireland has a right to peace of mind, to feel safe in their own homes and to be seen as equals."
He continued: "We consider that those in a leadership role have a particular responsibility to uphold respect for others and promote good relations across and within our communities.
"That means moving beyond a simple condemnation of violence, important as that may be."