Hamidullah Akhund was arrested "somewhere in Kabul in the past few days", ''New York Times'' reported quoting an Afghan Government spokesman Sifatullah Safi.
Few details were released about the suspect and the circumstances of the arrest, the Times said, as Afghan President Hamid Karzai appointed a high level team headed by the defence minister to probe the killing.
"We are getting close to the truth", the spokesman said, as the peace envoy's killing threatens fresh turmoil in the nation as Rabbani supporters are up in arms against continuing peace efforts with the Taliban.
Rabbani, chairman of the Afghan Peace Council was killed by a turban bomber who claimed to be a peace emissary from the Taliban leadership.
Police and other senior figures have blamed the Taliban for the attack, but unusually, the leaders of Afghanistan's ten-year insurgency have so far refused to comment on it.
The investigation will be led by Afghan defence minister Abdul Rahim Wardak, seen as close to Karzai, a statement from the presidential palace said.
A government official said that Akhund had spoken to the bomber a day before the attack. The bomber was believed to be a trusted Taliban emissary, had left the official guesthouse where he was staying to speak to the suspect.
President Karzai and his government have come under intense pressure from political rivals and the Afghan public to arrest whoever was responsible for the plot to assassinate Rabbani.