Turnbull said the Cabinet has agreed the titles are no longer appropriate in the modern awards system and the Queen has approved his request to scrap them.
"The cabinet recently considered the Order of Australia...and agreed that knights and dames are not appropriate in our modern honours system," Turnbull said in a statement.
"The Cabinet resolved to recommend to Her Majesty that she amend the Letters Patent, which establish the Order of Australia, so that Knights and Dames would no longer be appointed to the Order," he said.
The honours were reintroduced last year by former prime minister Tony Abbott. Australia had scratched the honorary titles in 1986 as anachronisms. The titles are granted for public service. "It is a long way from being the most important issue in Australia today," Turnbull said.
"This reflects modern Australia. Knights and dames are titles that are really anachronistic, they're out of date, not appropriate in 2015 in Australia," he added.
He said that awards in the Order of Australia were an important way of honouring the achievements and service of many Australians, including those unsung heroes who might not otherwise be recognised outside their local communities. Existing knights and dames, including Prince Philip, will retain their titles.