K.T. Thomas said that since Murugan, Santhan and Perarivalan had spent 22 years in prison, hanging them now would amount to punishing them twice for the same crime.
The three men are lodged at the Vellore jail in Tamil Nadu.
Thomas told reporters here: "If they are going to be awarded (death sentence) after spending 22 years in jail, it would be like giving them two punishment for the same crime, which is against the constitution."
He urged President Pranab Mukherjee to review his decision to reject their mercy petition.
"The president should reconsider this because as of now the three have undergone a punishment more than the term of life imprisonment," he said.
In 1999, a three-member Supreme Court bench comprising Thomas, Justice D.P. Wadwah and Justice S.S.M. Quadri awarded death punishment to Murugan, Santhan, Perarivalan and Murugan's wife Nalini.
Thomas dissented on death punishment to Nalini, whose sentence was commuted to life imprisonment after then president Pratibha Patil accepted her mercy petition.
While Perarivalan and Murugan are Indian citizens, Santhan and Murugan are from Sri Lanka.
A Tamil Tiger woman suicide bomber assassinated Rajiv Gandhi at an election rally near Chennai May 21, 1991.
All four listed in the case were charged with contributing to the killing. Among those wanted for the assassination were the Tamil Tigers chief Velupillai Prabhakaran, who was killed in Sri Lanka in 2009.