Reports had said the powerful military had been pressuring the civilian government to remove Haqqani since Pakistani-American businessman Mansoor Ijaz claimed he had drafted and delivered the memo to former US military chief Admiral Mike Mullen in May on the instructions of the envoy.
Haqqani was today forced out of office over his suspected role in the secret memo, which had angered the powerful army.
Haqqani, the former Pakistan Ambassador to the US told the media he had stepped down to "bring closure to this meaningless controversy threatening our fledgling democracy".
A "transparent inquiry" would "strengthen the hands of elected leaders whom I strived to empower", he added.
Pakistan's democracy was "far more important than any artificially created crisis over an insignificant memo written by a self-centered businessman", he said.
Haqqani had dismissed Ijaz's claims and said he was considering legal action against the businessman.
The memo made public by Ijaz said the Pakistan government had sought US help to stave off a military coup in the wake of the American raid that killed Osama bin Laden in the garrison town of Abbottabad on May 2.
The memo further said the government would revamp Pakistan's national security set-up by removing elements with links to militants.
Ijaz, the controversial businessman who triggered the crisis, welcome Gilani's decision to conduct an inquiry and said he would provide "full and transparent cooperation".
He added, "My help to ISI in their preliminary investigations played an important role in today's events."