"Pakistan and India relationships are peculiar. No benign neglect will work. Let's not let things drift," Kasuri said at an Indo-Pak seminar in Delhi organised by the centre for peace and progress.
He however, steered clear of talking about the Jadhav issue but regretted that 'things are not looking pretty good' between the two countries.
"What is happening in Kashmir and the news from Islamabad," he said referring to Jadhav and the recent deaths of eight civilians in the Kashmir valley on Sunday during a Lok Sabha by-poll.
Kasuri expressed hope that "PM Modi had a sense of history and he wishes to be part of the history in India-Pakistan relationship."
He said "It was impossible to think Pakistan could be isolated diplomatically and for argument sake if India succeeds it will give rise to new conflict. It is counterproductive."