Ninth seed Nadal, a 14-time Grand Slam champion, defeated the sixth-seeded Monfils 6-3, 6-3, 4-6, 6-4 on Rod Laver Arena in almost three hours and will play Canada's world number three Milos Raonic in the quarters.
It was a gutsy effort from the never-say-die Spaniard, who was down a service break with a fifth set looming before he broke Monfils twice to clinch victory.
The 2009 champion reached the quarter-finals in Melbourne for the ninth time and it was his first since the 2015 French Open.
"I'm very happy being in the quarter-finals of a Grand Slam after a couple of years without being there," he said.
"It's very special for me, especially here in Australia where I feel (it's) a little bit like home." He added: "Gael is a special player. He is able to play unbelievable shots. It is a pleasure to see him in the top rankings."
It was Nadal's first win over a top-10 player at a Grand Slam since he defeated number two Novak Djokovic to win the title at Roland Garros in 2014, his last Grand Slam win. He will go into his Australian quarter-final with a 6-2 lead over Raonic in their previous meetings.
Monfils was broken in his opening service game as Nadal went on to take the first set in 46 minutes. Nadal lost his service in the sixth game but broke the unpredictable Frenchman three times to open up a two-set lead and was closing in on a straight-sets victory.
But Monfils fought back, breaking Nadal in the ninth game and saving three break points in serving it out to trail 2-1 in sets. Nadal began to struggle in the fourth set as Monfils fought off two break points and broke the Spaniard in the next game with a net cord on one of his three break points.
But Nadal fought back, breaking the Frenchman with a marvellous backhand winner to level at 4-4 before holding serve to 5-4. Nadal then got to his first match point in Monfils' next service game, which was saved. On a second match point, the Frenchman's backhand was narrowly wide to send Nadal through.