"The prime minister has fallen well-short of what his constitutional and moral obligations are," Congress leader Anand Sharma told the media.
Referring to the September 28 lynching of a Muslim man in Uttar Pradesh following rumours that he ate beef, Sharma said Modi broke his silence only after President Pranab Mukherjee did.
"He spoke only after the president was forced to express his serious concerns over culture, values, ethos, particularly in regard to India's pluralistic society, democracy, co-existence and tolerance," he said.
"The prime minister makes people believe he has transformed India, which he has not. He has rather not been able to preserve India.
"Even constitutional guarantees are under assault," Sharma added.
The Congress leader's comments follow the widely denounced Muslim man's lynching, the killing of rationalists and the return of awards in protest by dozens of award winning writers.
Sharma said the situation in the country was causing "grave concern" to citizens who were "feeling insecure".
Sharma said Bharatiya Janata Party (BJP) president Amit Shah's reported pulling up of BJP leaders for making hate statements was "a cosmetic exercise".
"But nobody is going to be fooled," he said.
Reacting to Finance Minister Arun Jaitley's criticism of vandalism, Sharma called for "tangible action against elements, organisations and forces who have smashed India's name in the international community".
He said: "Those who have a divergent viewpoint from the ruling establishment or the BJP, the RSS and Sangh Parivar are targets of direct attacks.
"Even eminent Indians who have protested have not been heard but they have been humiliated."