Stormont Castle was evacuated on Tuesday morning after the package marked for Ms Villiers was discovered.
Army bomb experts made the viable device safe.
Police confirmed that the package was a viable letter bomb type device similar in make up to other devices found in recent days.
A PSNI spokesperson urged the public to remain vigilant when opening their mail and asked anyone who notices anything suspicious should contact them immediately.
Speaking from London, where she was meeting with Richard Haass, Ms Villiers branded the actions of the perpetrators "reckless".
"If those responsible think that this kind of criminal activity will further any agenda, then they are completely mistaken," she said.
I utterly condemn the attempted attacks we've seen over recent days.
While senior PSNI representatives warned the Northern Ireland Affairs Committee at Westminster that dissidents remained committed to violence.
PSNI Assistant Chief Constable Drew Harris told them:"There does not seem to be any diminution in their intent or any sense of them trying to find some route for dialogue with other parties or the Government."
Mr Harris expressed concern the extremists could get hold of more explosives and guns or make substantial amounts of money.
"We have concerns about upcoming anniversaries and whether they want to use those for their own purposes in getting publicity," he added.
Chief Constable Matt Baggott assured the committee members that intensive investigative efforts would go into finding those responsible.
The alert comes after two letter bombs were sent to PSNI Chief Constable Matt Baggott and another senior officer in Londonderry last Friday.
Another was posted to the Public Prosecution Service office in Londonderry on Monday.
Dissident republicans had been blamed.
Stormont Castle is the headquarters of First Minister Peter Robinson, of the DUP, and deputy First Minister Martin McGuinness, of Sinn Féin.
It is understood Mr Robinson was moved to Parliament Buildings during the alert and continued his work and meetings unaffected. Dozens of staff members were also evacuated.
Northern Ireland will not be dragged back by terrorists who have nothing but misery to offer.
The DUP leader said: "Those responsible for sending this, and other devices, through the post have absolutely no regard for the lives of postal workers and staff working in offices. They will not further any aim or objective by their vile and callous deeds."
Mr McGuinness was not at Stormont Castle at the time, as the Assembly is in recess. Few MLAs were at Parliament Buildings for this reason.
Meanwhile, Justice Minister David Ford has visited one of the Royal Mail postal sorting offices where a parcel bomb was discovered last week.
The device, addressed to Chief Constable Matt Baggott, was found by postal workers at the facility in Mallusk, Co Antrim, on Friday - and is being treated as a murder attempt.
Mr Ford thanked staff for their vigilance in handling suspicious packages.
He said: "There are delivery offices across Northern Ireland handling many thousands of letters and parcels every day.
"Royal Mail has been very active in putting in place security measures to detect and deal with suspicious packages, which in turn helps to secure and protect our citizens.
"I was grateful for the opportunity to personally thank some of the staff involved in dealing with our mail and carrying out those security checks. There is an obvious threat to these staff when reckless people abuse the mail system and I commend them for their vigilance."
UUP Justice spokesperson Tom Elliott said the incidents need to be addressed by Westminster.
"When one factors in the pipe bomb attacks on Police patrols in Newtownabbey and Londonderry as well as the recent security alerts in Lurgan, Londonderry and Belfast, it is clear that republican terror groups are trying to up the ante," the Fermanagh & South Tyrone MLA said.
"These fanatics may be small in numbers and have little popular support, but that does not prevent them from presenting a threat to life, not least to the postal workers and office staff who have to handle such devices when they are sent through the mail."
Alliance East Belfast MLA Chris Lyttle described the discovery of a fourth device as "an attack on democracy".
"Theresa Villiers is a democratically elected Member of Parliament and holds a key position within the Government, while those responsible for this device have very little support and whose only aim is to cause violence and disruption across Northern Ireland," he said.
"There is no place in our society for the reckless actions of those behind this letter bomb. Somebody could have been killed or seriously injured had this device exploded."
US diplomat Richard Haass, who is leading all-party discussions on solving a number of long-standing issues such as parades, flags and dealing with the past, said the "use of violence and terror for political aims (are) unacceptable".
"Northern Ireland has come too far, has too much potential for this," he tweeted.