Neil Quinn scored the vital goal in the 24th minute when he netted from close range after a Chris McFall throw-in.
The goal was enough to see the minnows pull off one of the biggest upsets in Irish Cup history, and consign Scott Young's Carling Premiership side to a hugely embarrassing defeat.
Elsewhere, Donegal Celtic needed a late equaliser to avoid defeat at the hands of Championship side Institute.
Stephen Parkhouse opened the scoring for Stute just before half-time, but Liam Bradley netted six minutes from time to secure the replay.
Coleraine left it late to earn victory over lowly Larne Tech Old Boys - the Bannsiders were 2-0 down in the closing stages before Paul Owens, Shane Jennings and Stephen Lowry netted in the last five minutes to spare the blushes of manager Oran Kearney.
Linfield survived an early scare against Ballyclare Comrades, coming back from a goal down to earn a convincing 7-1 victory and book their place in the sixth round.
Mark McClelland gave the visitors a shock lead in the 22nd minute, but a brace apiece from Phillip Lowry and Peter Thompson, along with a Rory Patterson hat-trick, ensured safe passage for David Jeffrey's side.
In the all-Premiership ties Glenavon and Ballymena United progressed at the expense of Portadown and Lisburn Distillery respectively.
Goals from Brian McCaul and player-manager Gary Hamilton proved enough to see the Lurgan Blues through at the expense of their neighbours, although Ryan McCluskey grabbed a late consolation for the Ports.
Hamilton also received a red card late in the game.
In the game at Ballyskeagh Gary McCutcheon put Ballymena ahead just ten minutes in, but his effort was cancelled out by a Gary Liggett strike in the 53rd minute.
However, the Whites' joy was short-lived as Alan Teggart grabbed the winner virtually straight from the restart.
All of the other Premiership sides successfully secured their place in the next round at the expense of lower-league opposition, with Carrick Rangers beating Killymoon Rangers 3-1, Cliftonville seeing off Ards 2-0, Crusaders overcoming Warrenpoint Town 3-0 and Dungannon Swifts securing a 3-1 victory over Larne.
There were also wins for Newry City, Derriaghy CC, Ballymoney United, Ballinamallard Utd and Newbuildings Utd.
Coagh United face a replay against Loughgall after their game finished 2-2.
JJB Sports Irish Cup 5th round results:
Banbridge Town 0 - 1 Newry City
Newry City: Grant 88 (penalty).
Bangor 2 - 4 Ballymoney United
Bangor: McLaughlin 39, 78.
Ballymoney UTD: Whitford og 6, McKenna 17, Kearney 23, Christie 90.
Coagh United 2 - 2 Loughgall
Coagh UTD: Scott 62, Black 79.
Loughgall: Hawthorne 66, Keenan 69
Dundela 0 - 3 Ballinamallard United
Ballinamallard UTD: McCartney 43, Sargeant og 53, Carters 91.
Glenavon 2 - 1 Portadown
Glenavon: McCaul 54, Hamilton 64, Hamilton sent off 90.
Portadown: McCluskey 83.
Newbuildings United 4 - 1 Dunmurry Rec.
Newbuildings UTD: Falconer 30, Scanlon 56, 62, Moore 78.
Dunmurry Rec.: Hands 83, Beggs sent off 59.
Cliftonville 2 - 0 Ards
Cliftonville: Clarke 86, Garrett 87.
Coleraine 3 - 2 Larne Tech Old Boys
Coleraine: Owens 86, Jennings 88, Lowry 90.
Larne: Fulton 32, Thompson 52.
Crusaders 2 - 0 Warrenpoint Town
Crusaders: Morrow 19, McKeown 46.
Dergview 0 - 1 Derriaghy CC
Derriaghy: Baker.
Dungannon Swifts 2 - 1 Larne
Dungannon: Friars 42, Lavery 57.
Larne: Maguire 30.
Glentoran 0 - 1 Newington YC
Newington: Quinn 24.
Institute 1 - 1 Donegal Celtic
Institute: Parkhouse 45.
Donegal Celtic: Bradley 84.
Killymoon Rangers 1 - 3 Carrick Rangers
Killymoon: Smith 26.
Carrick: McDowell 19, Lagan 39, 45.
Linfield 7 - 1 Ballyclare Comrades
Linfield: Lowry 29, 41, Thompson 35, 54, Patterson 45, 60, 75.
Ballyclare: McClelland 22.
Lisburn Distillery 1 - 2 Ballymena United
Distillery: Liggett 53.
Ballymena: McCutcheon 10, Teggart 54.