Polling stations in the Black Sea island republic will remain open from 8 a.m. to 8 p.m., Xinhua reported.
Monitoiring the referendum are 135 foreign observers from 23 countries and 1,240 local ones. The controversial vote also attracted some 2,500 journalists from around the world.
The final results of the referendum will not be announced until Monday. But an exit poll has been commissioned by the Kryminform news service.
Crimea held referendums in 1991 and 1994. This year's referendum asks voters whether they want to reunify with Russia or they want to remain within Ukraine with the peninsula's 1992 constitution restored to give Crimea greater autonomy.
The autonomous republic's authorities predicted that voter turnout would be at least 80 percent and an overwhelming majority would vote in favour of joining Russia.
On March 6, the Crimean parliament voted to become part of Russia and decided to hold a referendum Sunday on Crimea's future status.
The regional parliament Tuesday adopted a declaration of independence from Ukraine, which specified that Crimea would break away from Ukraine if around two million residents voted in favour of joining Russia in the referendum. But Ukraine's constitutional court has ruled the referendum was unconstitutional.
On Saturday, Moscow vetoed a UN Security Council draft resolution, drawn up by the US and backed by Western countries, that declared the referendum invalid. Russia would "respect the will of the Crimean people", said Vitaly Churkin, Russian ambassador to the UN.