Washington and Kabul had disagreed over the new Taliban office in Qatari capital Doha.
Karzai is now expected to continue security talks with the US, which he had suspended in protest over the office, BBC reported.
Karzai stated that the opening of the building contradicted earlier US security guarantees to Afghan government.
The Taliban premises in Qatar opened Tuesday, on the same day that NATO handed over Afghanistan's security to the Afghan government for the first time since the Taliban were ousted in 2001.
In a statement, Karzai Wednesday said that the Taliban office was "totally contradictory to the guarantees made by the USA to Afghanistan".
His officials said Karzai objected to a Taliban flag flying from the building, and also the name given to the building - the Islamic Emirate of Afghanistan.
He said Afghan negotiators would stay away from the Qatar talks until "foreign powers" allowed the process to be run by Afghans - and also stay away from the latest round of security talks with the US on the American presence in Afghanistan after Nato leaves in 2014.
But Karzai's spokesperson now says Kerry has informed the Afghans that the Taliban office is removing the flag and the sign designating the building as the Islamic Emirate of Afghanistan will be replaced by one saying Bureau of Peace Talks.
President Karzai told Kerry that once he saw Afghan and international networks showing evidence that these changes had been made, he would "be ready to keep the wheels rolling" on the process.
Now a meeting will be held in Kabul Thursday to discuss the next steps.