The veteran diplomat however did not approve India's cancelling talks with Pakistan in the wake of serial ceasefire violations on the Line of Control in Jammu and Kashmir.
Natwar Singh said: "Speaking as somebody who has spent a lifetime in foreign affairs, if I was Modi's advisor, I would have advised to postpone the secretary-level talks and not call them off.
"You have to start negotiating at some point of time. You can't always maintain a stand of no dialogue. It's a difficult relationship and Pakistan is going through a difficult time of their own making."
He also disapproved Pakistan cricketer-turned-politician Imran Khan challenging his own government.
"Pakistan is going through a difficult time where its own government is being challenged. Khan was a great cricketer but I don't think he is an astute politician," he said.
The veteran politician also said Modi will not abrogate Article 370 even if the Bharatiya Janata Party (BJP) has it in its manifesto.
He said: "Vajpayee didn't, Advani didn't and even Modi will also not abrogate Article 370.
"Governing Gujarat is different from governing the country. From Gandhinagar you have one view of India and from Delhi it's another. So he is gonna find out the facts of political life at the level of a prime minister."
Natwar Singh also hailed Modi's decision to scrap the planning panel, saying it was long overdue.