According to Rudd, he will leave parliament at the end of this week.
"For our family, recent statements since the September election (in Australia) have been particularly hurtful," Xinhua quoted him as saying.
"The decision that I have made has not been taken lightly, particularly given the big attachment I have for the community I proudly represented in this place these past 15 years," he said in the speech.
"But for me, my family is everything, always has been, always will be, which is why I will not be continuing as a member of this parliament beyond this week."
He also thanked the people of Australia for electing him as their prime minister.
"To have served as prime minister of Australia has been a great honour accorded to very few in our country's history," he said, as he wished current Prime Minister Tony Abbott luck, saying that the office of prime minister was the "hardest job in the land".
"The prime ministership of this Commonwealth is not easy; it is the hardest job in the land," he said.
"I wish Tony, his wife Margie and their family all the best for the rigours of high office that inevitably lie ahead," he said.
First elected as MP from Griffith in 1998, Rudd became prime minister in 2007, defeating then prime minister John Howard and taking the Australian Labour Party back to office.
But his time as prime minister came to an abrupt end in June 2010 when he was dumped by his party colleagues in favour of Julia Gillard.
Rudd reclaimed the job of prime minister in June this year when he defeated Gillard in a party-room showdown. But he went on to lose the Sep 7 election to Tony Abbott.