Sharif, is on a two-day visit to Karachi, the country's commercial centre, where targeted attacks claimed about 1,800 lives in the first six months of this year and economic activities have been badly affected due to violence.
The powerful group of ethnic Urdu-speaking Mutahida Qaumi Movement (MQM) is calling for handing over the city to the army to deal with the miscreants.
"I am deeply concerned over the violence in Karachi and will take action which can restore peace," Sharif told a gathering of leading businessmen from the city.
He said his government would give more powers to the paramilitary Rangers, which will have a major role in the proposed "targeted operation" against the miscreants. He said peace is essential for investment and economic progress.
Sharif: Peace essential for investment and economic progress
The Human Rights Commission of Pakistan (HRCP) said those killed in the city included victims of sectarian violence, targeted killings, as well as over 100 people whose dead bodies have been found and who appeared to have died in incidents of violence.
The prime minister discussed with political leaders and the businessmen various options to end the violence.
The government has already got support for the targeted operation against those who are behind the killings, kidnapping for ransom, extortion of money from the businessmen and occupying land.
"The operation will be without any discrimination to clear the city of the criminal elements," the prime minister said.