"We have told them (the US) to leave the airbase," Mukhtar said during an interaction with Pakistani journalists at his office in the garrison city of Rawalpindi.
Mukhtar's comments came in the wake of the Pakistani Army's demand that the strength of US military personnel in the country should be reduced to the "minimum essential".
Scores of CIA operatives and American military trainers have left Pakistan in recent weeks.
The Pakistani military made the demand in the aftermath of the US raid that killed Al Qaeda leader Osama bin Laden in the garrison city of Abbottabad on May 2.
Numerous reports have said that the CIA operates its unmanned spy planes from Shamsi airbase in Balochistan province, 900 km from Islamabad. Images of US Predator drones at the airbase have been published by Google Earth.
Secret American diplomatic cables leaked by WikiLeaks have suggested that the United Arab Emirates, which has controlled Shamsi airbase since the 1990s, has allowed the US to use it for its drone flights.
Referring to the US raid against bin Laden, Defence Minister Mukhtar said trust between the US and Pakistan has been "reduced to a great extent" after the incident. "This trust deficit could be reduced by sitting together and taking joint actions," he said.
The US has stopped providing reimbursements from its Coalition Support Fund for Pakistan's expenses on the war against terror in the tribal areas and, as a result, "this war was harming our economy", Mukhtar said.