The fighters, who had been hoping to mop up the last pockets of resistance in two northwestern residential districts, withdrew to the police headquarters they had captured on Tuesday, said an AFP reporter.
Commanders of the forces loyal to the National Transitional Council (NTC) said the Gaddafi diehards were cornered within about two square kilometres of the Mediterranean city.
As heavy artillery fire was heard in the city's west and thick black smoke rose over the waterfront to the north, ambulances with sirens ablaze ferried the many wounded out for treatment.
Four pro-NTC fighters were killed, including two by friendly fire, and another 40 were hurt, mostly by sniper fire, said Rawad Friwan, a surgeon at a field hospital on Sirte's western outskirts.
"Earlier in the day, we had been engaged in street fighting, but we have stopped. The pro-Gaddafi fighters have been firing rockets, mortars and bombs at us," said Fayisal Ahmed Bringo, a new regime fighter.
"There are still 500 pro-Gaddafi fighters in Sirte and our forces today arrested 15" of them, he said.
The intensity of fighting eased later to machinegun and occasional rocket fire, mainly from the forces of the new regime, as they waited for further orders.
Sirte is a key goal for Libya's new leaders who have said they will not proclaim the country's liberation and begin preparing for the transition to an elected government until the city has fallen.
The new regime began its siege of Sirte on September 15 before launching what it termed a "final assault" last Friday that has seen at least 95 of its troops killed and hundreds wounded, according to medics.
Its forces have encircled Gaddafi loyalists after taking control of Sirte's waterfront, its showpiece conference centre, university, hospital and main square.
A top adviser of NTC chief Mustafa Abdel Jalil meanwhile backtracked on his announcement that they had captured Kadhafi's feared son and national security chief Mutassim in Sirte, after it was denied by military commanders in the city.
NTC oil minister Ali Tarhuni vowed Libya would investigate "every penny" of suspicious oil contracts signed under Kadhafi's regime, which was responsible for what he called "unbelievable" corruption.
"There will be specialised committees that will look into all these contracts and agreements starting with the oil sector," Tarhuni said, without giving details on contracts or companies.
Libya's oil production, which collapsed after the uprising in February, is expected to rise to nearly one million barrels per day by April from the current 400,000, said Nuri Berruien, head of the state-run National Oil Company.
Italian energy company ENI said it has resumed supplies of natural gas through the Greenstream pipeline linking Libya and Italy, following an eight-month suspension due to the conflict.
Berruien, confirmed the supply of "limited quantities" to Italy but said "official" exports would not resume until the end November or early December after levels are agreed.
Elsewhere, the NTC said it signed an agreement with NATO opening up parts of Libya's airspace to commercial flights, despite a UN no-fly zone that cleared the way for air strikes on Gaddafi's war machine.