Fire service officials said managing director of the Tung Hai Sweater factory Mahbubur Rahman and his friend Additional Deputy Inspector General of Police ZA Morshed were among the dead as the fire broke out on several floors of the 11-story building housing the factory and residential complex late last night.
"The casualty was less as the factory was closed when the fire broke out... the victims were having chats at the managing directors office," a supervisor of the factory told reporters at the scene.
Fire officials said that apparently ignited from an electric short circuit, the fire broke out on the second floor of the building and quickly spread on the third and fourth floors where the factory was housed.
A doctor at the state-run Dhaka medical college hospital said they declared dead five people as they were brought to the facility while they appeared to have suffered also from suffocation caused by the fire.
The incident comes nearly two weeks after a building housing garment factories collapsed in Bangladesh, killing nearly 900 people.
Bangladesh has closed down 18 garment factories temporarily as part of its efforts to allay fears of international buyers and rights groups over safety and labour standards in the country's beleaguered apparel sector.
"We will close down factories deemed to be dangerous," Textiles and Jute Minister Abdul Latif Siddique said in a news briefing on Wednesday.
Siddique said among closed factories 16 were in Dhaka and two in south-eastern port city of Chittagong.