"There is something ahead of 'samata' (equality) and that is 'samarasata' (social harmony). 'Samarasata' needs a larger social awakening. 'Samata' can come through laws too," he said.
Modi declared that India has begun its journey towards social harmony.
Speaking at a function to mark the 152nd birth anniversary of Dalit icon Ayyankali, the prime minister said there was a need to create a system which ensures there was no injustice to anyone.
"We have to create systems where there is no injustice against anybody," he said.
The prime minister said equality has been guaranteed by the constitution, but equality alone cannot achieve social harmony.
Modi said one of the reasons why India's cultural heritage has been preserved for thousands of years is that in every age, social reformers have emerged from within to struggle against social evils and end them.
The prime minister also dwelt upon the contribution of Ayyankali and Narayana Guru towards education in Kerala.