The 25-year-old was hit by a car and then attacked by two men with a meat cleaver and a knife.
The vicious assault happened during the day, and eyewitnesses described the killing as "crazed".
Five people have been arrested on suspicion of conspiracy to murder the father of two.
Mr Nesbitt, Ulster Unionist MLA, said people across the UK were shocked at the soldier's murder, and urged councils to act in solidarity with the Rigby family.
He said: "By signing a book you're sending a message to the family, you're sending a message to the Armed Forces and you're sending a very powerful message to terrorism that we will not have it on the streetss of the United Kingdom.
"When it happens in the United Kingdom, no matter where, it is, I think important for the people of England, Scotland, Wales, and Northern Ireland, to come together in a united way and to stand firm together."
I think a lot of people would welcome the opportunity to go somewhere, such as Belfast City Hall, at the time of the funeral if they can, just to stand for a moment in solidarity with the family.
On Sunday, Drummer Rigby's family visited the scene of his murder
His widow Rebecca, mother Lyn, 46, and stepfather Ian, 54, laid flowers at the Woolwich Barracks where the soldier with the Royal Regiment of Fusiliers was based, and where hundreds of floral tributes have already been left by well wishers.
Mr Nesbitt wants all 26 councils across NI to remember Drummer Rigby, whose funeral will be held later this week.
He said providing space for people at the time of his funeral would be "an entirely appropriate gesture to make".
Mr Nesbitt added: "I would find it very difficult to believe that anybody would object to voicing their utter rejection of what happened last Wednesday in Woolwich."