It is the fifth draw from eight games for O'Neill, who was appointed over a year ago, while Northern Ireland remain without a win since beating the Faroe Islands in August 2011.
Gareth McAuley put the ball in the net in the 18th minute at Ta'qali National Stadium but it was disallowed for pushing.
Meanwhile goalkeeper Alan Mannus managed two very good saves to deny former Coventry striker and Malta captain Michael Mifsud.
Mannus started in goal with regular keeper Roy Carroll absent through illness.
O'Neill admitted his side were not good enough in the final third.
"It was a difficult game in that the conditions weren't particularly conducive to trying to play. The pitch was difficult, very bumpy, the wind obviously didn't help matters," he said.
"But overall, whilst we never really seemed in any great danger and we controlled large portions of the game, we didn't create enough chances.
"You want to win games, simple as that," O'Neill added.
"We've had eight games, we've had five draws and three defeats. We've had some very difficult games in there."
They were very well organised and at times we played into their hands a little bit with our choice of passing. I think at times a little bit of our inexperience in certain areas of the team showed up tonight.
O'Neill selected his team with next month's World Cup qualifying match against Russia in mind.
Chris Baird and Kyle Lafferty are both suspended for that game and did not travel to Malta.
There was a Northern Ireland debut for Alex Bruce in the holding midfield role while Walsall's Will Grigg won his second cap up front.
Bruce, who plays under his father Steve at Hull, previously played for the Republic of Ireland but was able to switch allegiance as he did not make a competitive appearance.
"I was pleased with Alex," O'Neill said of his performance. "I saw him play on Saturday against Millwall in that holding role and he played very well. It was important we saw Alex in there tonight."
It was the hosts who made the stronger start and Northern Ireland were almost behind in the fifth minute when Andrew Cohen crossed for Andre Schembri, who should have scored but shot straight at Mannus from eight yards.
Grigg fought off competition from Scottish Premier League top scorer Billy McKay among others for the striking role, but he blew his first big chance in the 17th minute when his heavy touch allowed keeper Andrew Hogg to block.
Moments later McAuley headed in a Chris Brunt corner but Bulgarian referee Nikolay Yordanov ruled it out.
Malta were looking the more likely to break the deadlock and Clayton Failla was the next to threaten Mannus' goal with a fierce free-kick from 30 yards that was a yard over the bar.
Wingers Niall McGinn and Shane Ferguson were having little joy down the flanks but in the 36th minute McGinn made progress down the right and Steven Davis crossed for Grigg to glance a header wide.
O'Neill made one change at half-time, Craig Cathcart replacing Jonny Evans in the centre of defence.
The visitors almost caught Malta cold as McGinn broke through in the middle but Hogg was off his line quickly to punch clear, while Aaron Hughes then did well to get to the byline on the right and pull the ball back but McGinn scuffed his shot.
In the 56th minute Mannus was called into serious action for the first time when Mifsud unleashed a stinging shot from the right, the keeper palming it away.
In the 63rd minute Inverness striker McKay was introduced, winning his first cap when he replaced Ferguson on the right, with Grigg remaining the lone striker.
The visitors were pressing but in the 70th minute they were indebted to Mannus for an excellent stop to keep out Mifsud once again.
Pat McCourt and Josh Magennis were introduced late on but there was no way through the Malta defence and the visitors had to settle for a disappointing draw.
It is only the second time the side failed to win at the Ta'qali in seven meetings.
Northern Ireland haven't score an away goal in a friendly since beating Finland 2-1 seven years ago.