The Sinn Féin MLA spoke as he visited the Short Strand in east Belfast on Wednesday.
There were violent scenes near the interface on Saturday as loyalist crowds returning from a city centre flag protest clashed with nationalist residents. Trouble erupted again on Monday night, amid reports that local homes were attacked on both sides of the community.
Mr McGuinness said he was there to "stand in solidarity with what is a small, isolated but very proud Catholic community" that has "borne the brunt of sectarianism" in recent weeks.
"We need to make it clear that we are not going to bow the knee to so-called loyalists or so-called republicans," he said.
"We need to be seen to be giving leadership. We need to be seen to stand together.
"We need to be seen to stand with the PSNI and the PSNI need to be seen to stand with the community.
"They do that knowing that they have the full support of our political leaders and they have our full support in bringing those people who have confronted our institutions, who have confronted our peace process over the course of the last number of weeks, before the courts."
The presence of Martin McGuinness here today has inevitably led to questions as to why he and the First Minister have not been seen together since the trouble began.
UTV's Tracey Magee
"I think this is an occasion where we do need to be seen to be standing together," Mr McGuinness told UTV.
The senior Sinn Féin politician said the trouble was being caused by a "very tiny minority who have obviously shown themselves capable of catapulting themselves into world headlines with a very negative message".
"Those people have to be defeated," he said, "and the only way they can be defeated is by resolute action by the political leaders and police."
"As I stand front and centre against the activities of violent so-called republicans, we also need to be seen to be standing together against those anti democratic, violent so-called loyalists who are trying to confront our institutions and damage the political progress that has been made," added Mr McGuinness.
Much of the violence that has broken out following weeks of flag protests has been concentrated in east Belfast.
NI Secretary of State Theresa Villiers was on the other side of the interface on Wednesday to meet with community and church leaders.
She called for politicians to tackle sectarianism.
Ms Villiers added it was important for all political representatives to be forthright in condemning violence and calling for real progress on building a shared society.
"I am encouraging them to continue to work together and send out a signal strongly that Northern Ireland is open for business," she said.
"They need to make progress on unfinished business for some in Northern Ireland.
"There remains deep-seated divisions on sectarian lines. It is hugely important we have bold moves to try and address that."