The collapse of the Wei Guan building, the most seriously damaged, has killed 31.
At about 1:50 p.m. local time, rescuers pulled from the Wei Guan building a man in his 20s, who later died. A six-month-old girl was rescued but died. Her father was also among the deceased.
More than 600 rescuers, including fire fighters and volunteers, as well as 115 fire engines and 44 ambulances, are racing against time to save 119 people who are still trapped in the rubble of the building. A total of 309 people have been rescued so far.
Previous reports said more than 460 people were injured.
The island's high-speed rail lines resumed full operations on 5:00 p.m. local time.
In the historic city of Tainan, eight buildings collapsed and another five were damaged.
A 6.7-magnitude earthquake hit Kaohsiung on Saturday at a depth of 15 kilometers, the China Earthquake Networks Centre said. The quake occurred two days prior to Spring Festival, the lunar new year.