"This is the hardest decision I've ever made and I don't know what I'll do next," Efe quoted Key as saying. He also resigned as the National Party leader.
The country's presumptive next leader paid tribute to Key "as one of New Zealand's greatest leaders".
"Through good times and bad, his strong leadership has been steadfast and this is a more confident, successful and self-assured country because of his contribution. He has truly made a difference," Xinhua news agency quoted Deputy Prime Minister Bill English as saying.
According to the media, Key's wife Bronagh asked him to step down due to the "extraordinary levels of intrusion" in the lives of their two children, daughter Stephie and son Max.
Key's announcement shocked fellow MPs of his centre-right National Party as much as it did to the rest of the country.
The National Party caucus will choose a new leader on December 12 and Key said he would back English if he put himself forward.
English was appointed the National Party leader in 2001 but resigned after he led the party to a disastrous election defeat in 2002.
Other senior National Party cabinet ministers being touted as potential contestants include Economic Development Minister Steven Joyce, Climate Change Issues and Associate Finance Minister Paula Bennett and Police and Corrections Minister Judith Collins.
Political opponents offered mixed views, with the leaders of the Labour Party and Green Party praising Key for his service to the country.
"Key has served New Zealand generously and with dedication," Labour Party leader Andrew Little said.
Green Party co-leader James Shaw said Key should be applauded for his commitment to public service and to the country.
The New Zealand First party was less magnanimous, saying Key's reasons for standing down could not be credible.
"The fact is that the economy is not in the healthy state that the Prime Minister has for so long claimed and there are other issues which have caused this decision as well," party leader Winston Peters said.
The possible date for new elections in New Zealand is November 2017.