He conveyed this directly to Pakistan President Asif Ali Zardari in response to his invitation in July, sources said.
In a letter to Singh, Zardari had invited him to undertake the visit to his ancestral place Gah in Punjab on the occasion of Guru Nanak Jayanti on Nov 28.
Zardari had suggested that if Singh's visit coincided with celebrations to mark the birth anniversary of Guru Nanak, it would be well received by the Pakistani people and reinforce the desire of both countries to promote inter-religious harmony.
"Our invitation for you to visit Pakistan is outstanding. Your visit will provide us an opportunity to take stock of our bilateral relations," the Pakistan President had written.
The sources said that while India was willing to do business with Pakistan in aspects like economy and commerce, it would like to hold it responsible for not much progress on the issue of punishing those behind Mumbai attacks of 2008.
India does not feel that the relationship would see any major transformation. "What we are looking for is just normal relationship. It will have its costs and benefits," a source said.
The comments assume significance as these come ahead of the planned visit of Pakistan's Interior Minister Rehman Malik here next week.
India is also working on plans to allow Pakistani judicial commission to visit here in connection with the Mumbai attacks.
India has conveyed to Pakistan that the issue would be considered positively and it is working out aspects like timing and terms under which the commission would take place.