In a video posted on YouTube, a member of Co Kildare band The Druids appears to call for British soldiers and their "Orange comrades" to go back to England on the final night of the festival.
Organisers say the comments were "wrong", "regrettable", and "disappointing".
They say they "do not reflect the views of the organisers of the Ardoyne Fleadh".
The DUP has branded the comments as "hate speech" and a "glorification of terrorism" and called for public funding to be withdrawn.
The party also called for the festival's sponsors to disassociate themselves from the incident.
Councillor Lee Reynolds said: "This turn of events came as no surprise to the DUP and was one of the reasons why we have opposed the licensing of this event.
"The DUP has submitted a formal complaint to the PSNI about the anti-British and anti-Protestant hate speech and the glorification of terrorism at this event."
This type of behaviour should be considered justifiable grounds for the withdrawal of all public funding.
Lee Reynolds, DUP
Organisers have pledged to review guidelines in relation to performers and acts following the incident "to ensure that this does not happen again".
The Chairperson of the Ardoyne Fleadh Cheoil Eddie Copeland said: "We regret any offence caused by this incident.
"It was wrong, regrettable, disappointing and should not have happened.
"The Ardoyne Fleadh is a community festival that has been running for 24 years.
"We are a totally voluntary group who struggle yearly to raise the funds necessary to make the event happen which gives thousands of local people a positive outlet over the tense summer months."
Earlier Sinn Féin Councillor Jim McVeigh said that the sectarian comments made at the event were "wrong".
He said: "(The festival) has attracted thousands of people from the local community and beyond to its open air concerts and to a packed programme of community events.
"There is currently a police investigation ongoing into an alleged sectarian comment made from the stage of the Fleadh on Sunday night.
"Sectarianism is wrong and it has no place at music or cultural events."
He added: "We all have to be careful about the language we use.
"The remarks about the Orangemen were inappropriate, the other remarks about the British Army were fair enough.
"That's fair political comment as far as we are concerned. We want to see the army withdrawn from this part of Ireland."
Councillor McVeigh said the event was hugely popular, attracting thousands of people each year.
"This has a broad agenda, it's not just rebel nights. It's a constructive event and it is the type of thing that needs to be promoted.
"And I would like to see more funding for the festival."
These comments do not reflect the views of the organisers of the Ardoyne Fleadh or the people of Ardoyne.
Eddie Copeland, Chairperson of the Ardoyne Fleadh Cheoil
SDLP North Belfast MLA Alban Maginness said: "Many of us are trying to create a city of reconciled citizens at peace with themselves. The SDLP strongly condemn republican and loyalist intolerance.
"It would, however, be incredibly short-sighted to end an entire community festival which encompasses art, sport and political debate based on the actions of one group."
Alliance Councillor Nuala McAllister also voiced her concern.
"This type of inflammatory language only serves to keep Northern Ireland stuck in the past, entrenched firmly in the tribal lines that define our troubled history," she said.
Ulster Unionist Belfast councillor, Alderman David Browne, said: "The fact that these were made in an interface area where we are all trying to build community relations makes it even worse."
"Apologists for this type of disgusting behaviour should remember that we are living in the year 2014 and not the dark days of the Troubles."
Meanwhile, Mick O'Brien from The Druids told the Belfast Telegraph the comments were "taken out of context" and that they aspire to a united Ireland "brought about by peaceful means".
Police said they were investigating the complaint.
A spokeswoman for the PSNI said: "PSNI have received a complaint and enquiries are ongoing."
Belfast City Council has said it did not provide funding for the event.
The Community Relations Council (NICRC), which gave this year's Ardoyne Fleadh a £5,000 grant towards infrastructure costs and marketing, says it is aware of the incident.
"The grant was subject to a number of conditions one of which was that it should be used to support the values and aims of the Community Relations Council," a spokesperson said.
"The CRC will be following its usual process in these circumstances by asking the event organiser to respond to the reports to determine whether the conditions of the grant have been complied with."
The body said no element of the grant was allocated "towards the cost of performers".