"He died peacefully in his home, surrounded by his family... of an illness called myelodysplastic syndrome," said a relative who spoke on condition of anonymity.
Gelman passed away shortly before 4:30 p.m. on Tuesday.
"The wake will be Wednesday," the source said, adding that "there will not be any official ceremony."
The Argentine government declared three days of mourning.
Gelman, a native of Buenos Aires, received numerous honours over the course of his career, including Mexico's Juan Rulfo Prize for Latin American and Caribbean Literature, Spain's Queen Sofia Prize for Ibero-American Poetry and the Cervantes Prize, the most prestigious literary award in the Spanish-speaking world.
He was suffering from an illness called myelodysplastic.
The poet fled Argentina in 1975 because of death threats from a right-wing death squad and later lost his son and daughter-in-law to the military junta that held power in Buenos Aires from 1976-1983.
He spent time in Europe and Nicaragua before settling in Mexico in the late 1980s.
At a 2012 ceremony where he accepted the Mexican Fine Arts Medal, Gelman thanked Mexico for its generosity toward the "thousands of refugees" who arrived in the country over the past few decades.
He went on to describe himself as an "Argenmex, which is how the thousands of Argentines who found refuge in Mexico self-baptized themselves."
Gelman's daughter-in-law was pregnant at the time of her abduction by Argentine secret police and the girl she gave birth to in captivity was given in adoption to the family of a Uruguayan police officer, but the poet managed to locate his granddaughter in 2000.
He published his first volume of poetry, "Violin y otras cuestiones," in 1956.