"There is no conspiracy called 'Love Jihad'. Such rumours are being spread to polarise votes," Madani said in an interview to TV Today Network.
"I would urge Modi to stop such rumours from spreading... And if it is true, those guilty should be given tough punishment," he said.
He, however, added that both Hindus and Muslims in India "do not tolerate marriages outside their religion or caste".
Asked about the stand taken by some BJP leaders that all living in India are "Hindu", Madani said: "I would say every Indian is 'Hindi'... No one has problem in being called Hindi Muslim, or Hindi Christian".