Of the 66 lawmakers present during the parliament session, 36, including from Norov's pasrty, voted for his dismissal, Xinhua reported.
Norov was criticised for incompetence and cronyism and corruption by the opposition Mongolian People's Party and some from within his Mongolian Democratic Party.
At present one of Norov's senior aides is under investigation by the country's anti-corruption body, Independent Agency Against Corruption.
MPP lawmaker Khayankhyarvaa Damdin was quoted by Xinhua as saying, "because of the policy mistakes of (the) so-called reform government, the country's economic growth, which was 17.5 per cent two years ago, decreased every year and now it looks it will be 6.3 per cent."
Altankhuyag is also being blamed by the opposition for allowing embezzlement by his relatives and aides.
Now, Norov's senior deputy, Terbishdagva Dendev, will serve as acting prime minister before a new premier is appointed and all current ministers will work as acting ministers.
(With inputs from agencies)