This is the prime minister's first visit to the kingdom since King Salman assumed the throne last month.
The new Saudi monarch had invited the Pakistani prime minister as part of his consultations with Muslim leaders on several issues faced by the Islamic countries.
Turkish and Egyptian leaders have already visited Saudi Arabia.
Saudi Arabia last week hosted an international conference of Islamic scholars to explore ways to fight against terrorism and violent extremism in Islamic countries.
Sharif will hold talks with the Saudi leaders on bilateral and regional issues, focusing on anti-terror cooperation, Xinhua news agency reported citing officials. He will also meet members of the Pakistani community in Saudi Arabia, the foreign ministry said.
The prime minister is accompanied by a high-level delegation that includes Finance Minister Ishaq Dar, Chief Minister of Punjab province Shahbaz Sharif, Special Assistant to the Prime Minister on National Affairs Irfan Siddiqui and Special Assistant to the Prime Minister on Foreign Affairs Syed Tariq Fatemi.
In his pre-departure statement, Sharif said that the two countries have grown stronger with the passage of time.
"Pakistan and Saudi Arabia are brotherly countries which are tied in bonds of shared faith and values," a statement from the PM Office quoted Nawaz Sharif as saying.