The oath of office and secrecy was administered in Hindi to Modi by President Pranab Mukherjee.
There was a certain poignancy to the momentous occasion as the inauguration of a new prime minister happened exactly 50 years after the death of India's first prime minister, Jawaharlal Nehru.
Present on the occasion of the change of guard was the outgoing prime minister Manmohan Singh, who remained 10 years at the helm when the general election left the Congress battered and reduced to its lowest tally of 44 seats in the Lok Sabha.
A council of ministers, numbering around 45, also took oath.
Among the foreign leaders who attended the ceremony at the forecourt of the presidential palace were Pakistan Prime Minister Nawaz Sharif, Afghanistan President Hamid Karzai and Sri Lankan President Mahinda Rajapaksa.
Also present were Mauritius Prime Minister Navinchandra Ramgoolam, Nepal Prime Minister Sushil Koirala, Prime Minister Tshering Tobgay of Bhutan and Bangladesh parliament Speaker Shirin Sharmin Chaudhury.
Congress president Sonia Gandhi, her son and party vice president Rahul Gandhi and former presidents A.P.J. Abdul Kalam and Pratibha Patil were among the over 4,000 people who gathered on the occasion.