Three men and two women, who escaped from the Lindt Chocolat Cafe, told media that the man had warned them that two bombs were set to explode in the cafe, and other bombs were planted at two locations in the city, Xinhua reported.
They escaped victims said the man forced his captives to call him "brother" and would release a hostage if the flag was delivered.
The hostage drama began at 9.40 am, and involved hundreds of armed police who had surrounded the cafe located at the Martin Place in Sydney's central business district. [Sydney siege: An Indian IT professional working with Infosys among hostages]
The hostages have been forced to hold a black and white Islamic flag on the cafe window that says, "There is no god but Allah, Muhammad is his messenger."
Shop staff in aprons could be seen with their hands on the windows.
Nearby streets were cordoned off and all trains were shut down near the area.
The police also evacuated the Sydney Opera House and shut down traffic on the Sydney Harbour Bridge.