Obama suggested Russia put "a concrete response in writing" and the two leaders agreed that US Secretary of State John Kerry and his Russian counterpart Sergei Lavrov would meet to discuss next steps, Xinhua quoted the White House as saying in a readout of the phone talk.
Kerry presented Washington's proposal again to Lavrov earlier this week when they met in The Hague on the sidelines of a nuclear summit in the Dutch city.
Obama urged Putin to support the constitutional reform and elections underway in Ukraine and "avoid further provocations", including the buildup of forces on its border with Ukraine.
The American leader also voiced continued US support for "a diplomatic path" in close consultation with the Ukrainian government with a view to de-escalating the crisis.
Obama "made clear that this remains possible only if Russia pulls back its troops and does not take any steps to further violate Ukraine's territorial integrity and sovereignty," the White House said.
He reiterated Washington's strong opposition to the actions already taken by Russia to "violate Ukraine's sovereignty and territorial integrity."
On March 21, Putin signed into law the treaty formally making Crimea part of Russia, five days after the autonomous region of Ukraine voted overwhelmingly in favour of joining Russia in a referendum.
Russia's next move is a top concern of the West, though Moscow said it had no intention to split Ukraine. But Russia also spoke of its responsibility to protect Russian speakers in eastern Ukraine from attacks.
Speaking on CBS's "This Morning" show on Friday, Obama urged Russia to pull back troops from the border with Ukraine and start direct talks with its neighbour, as US intelligence agencies have reportedly cited "mounting evidence" about Moscow's possible invasion of eastern Ukraine.
Earlier this week, Russia dismissed reports that it was mobilising troops to its border with Ukraine.
The Ukrainian crisis dominated Obama's agenda during his trip to Europe this week, in which he had threatened tougher economic sanctions in response to Russia's next move in Ukraine, including targeting its energy and other key sectors.