Tillerson was approved to be the top US diplomat in a 56-43 vote, with three democratic and an independent lawmakers joining all Republicans to back him, Xinhua news agency reported.
The confirmation came despite some lawmakers' skepticism over Tillerson's stance towards Russia, given his previous business ties with Russia and relationship with the country's leadership.
Senator Marco Rubio, who had a testy exchange with Tillerson during his confirmation hearing last month, noted that Secretary of State "is the most important cabinet position that the President has to nominate".
"There is so much uncertainty and debate about our role in the world these days," he said. "A lot of our allies have questions. Our adversaries are obviously watching very closely."
At the confirmation hearing, Tillerson called Russia a "danger" to the United States and said he favoured maintaining US sanctions against Moscow.
"Russia today poses a danger, but it is not unpredictable in advancing its own interests," said Tillerson. "Our NATO allies are right to be alarmed at a resurgent Russia."
He also recommended a "full review" of the nuclear deal with Iran, but he did not call for an outright rejection of the accord.
On the South China Sea issue, Tillerson told the hearing that China should stop island building in the South China Sea and be denied access to these islands, which China regards as an integral part of its territory.
Tillerson was widely known for opposing sanctions against Russia in his ExxonMobil corporate life. But he refused to label Russian President Vladimir Putin a war criminal as he had known him since the 1990s and was awarded Order of Friendship in 2013 by Putin.