Pakistan's Ambassador to the US Sherry Rehman said the remarks have further reduced the space for narrowing bilateral differences that have been hit by a series of conflagrations in the recent past.
"This kind of public messaging from a senior member of the US administration is taken very seriously in Pakistan and reduces the space for narrowing our bilateral differences at a critical time in the negotiations," Rehman said in a statement.
"It adds an unhelpful twist to the process and leaves little oxygen for those of us seeking to break a stalemate," she said.
Panetta had said in Kabul yesterday that the US was losing patience with Pakistan on the issue of militant safe havens on the Pakistani side of the border with Afghanistan.
Panetta's remarks on the need for Pakistan to do more to tackle militancy and extremism, made during a visit to India and Afghanistan, have irked the Pakistan government.
While in New Delhi, Panetta said the US would continue drone strikes against militants in Pakistan's tribal belt despite protests from Islamabad that the attacks violate its sovereignty.
Panetta's remarks came at a time when the two countries are set to resume key negotiations on ending a six-month blockade of NATO supply routes to Afghanistan. Pakistan closed the supply lines after a cross-border NATO air strike killed 24 of its soldiers in November.
The two sides have been unable to reach a deal on the routes due to various reasons, including the fees to be paid for every NATO container and tanker passing through Pakistani territory.
Peter Lavoy, US Assistant Secretary for Asia and Pacific Security Affairs, is scheduled to resume negotiations on new terms of engagement with Pakistan and the reopening of NATO supply routes today. During his two-day stay in Pakistan, Lavoy will meet senior civilian and military leaders.
The US and Pakistan have been involved in several issues of confrontation over the last year, including the issue of a CIA contractor who shot dead two people in Lahore, a covert US raid that killed Osama bin Laden, and the trial of a doctor who helped CIA find bin Laden.