The wire mesh structure is being put in place by the Department of Justice, and can be deployed and taken down by police when necessary.
It is designed to catch stones or other missiles thrown from both sides.
Justice Minister David Ford said: "It will be brought out if the police advise that it is required at particular times, a few days of the year, and taken away when it is not needed."
Trouble flared earlier this year between communities in the mainly unionist Newtownards Road and the mainly nationalist Short Strand, close to the Catholic church.
Mr Ford said it is "depressing" that a new barrier is needed, but said it shouldn't be described as a 'peace wall' as the only permanent structure will be a set of posts.
"It literally will be like a set of curtains so all the permanent fixtures will be is posts and the screen there a few days in the year," he continued.
"It's not in that sense a divide, it's not permanent and it's not a wall, but it is unfortunate."
Mr Ford said he is not sure about the exact timing of when the structure will go up in the church grounds but said it is not linked to the anniversary of the flag protests.
He added: "I'm not sure exactly the timing it will be put up but it's not particularly aimed at flag protests - on 12 July there was a bit of difficulty in the area with stone throwing."
Local councillor Niall Ó Donnghaile said it is "unfortunate" that the fence is needed.
The Sinn Féin man continued: "The fence is being erected after direct engagement between residents and the Department of Justice (DOJ), which was facilitated by Sinn Féin after residents demanded that action be taken to protect them and their families.
"We will continue to work with local residents, the DOJ and political and community leaders from within the PUL community to ensure we all move to a position where residents are able to live in peace, in secure and appropriate surroundings."