But Phelps knew deep down that he could have done more four years ago, and that "haunted" feeling made it impossible for his post-London retirement to stick.
"I never wanted to have that 'what if,'" Phelps said of his decision to mount a fifth Olympic campaign in Rio de Janeiro, at the age of 31.
"This journey has been incredible. Being able to fall in love with the sport again is something that I've always wanted to do, and I did it on my terms."
However, his comeback has been far from smooth -- disrupted by a drink-driving arrest that sparked a painful period of self-examination. Phelps has emerged stronger, his renewed relationship with his estranged father resonating even more after the birth of his own son, Boomer, with fiancee Nicole Johnson in May.
"It's a cool thing for Nicole and I, for where we are in our lives," Phelps said.
His relationship with long-time coach Bob Bowman endures, with Phelps voicing absolute trust in Bowman's ability to prepare him for his last Olympic hurrah. He has a chance to make more history as he seeks a fourth straight gold in the 100m butterfly and 200m individual medley.
The only Olympians to win four consecutive titles in the same individual event are athletes Al Oerter in the discus (1956-68) and Carl Lewis in the long jump (1984-96).
And he would love to avenge his 2012 loss to South African Chad le Clos in the 200m fly -- the first event Phelps ever swam in the Olympics -- in Sydney in 2000.
'This is it'
At the peak of his career, Phelps said he was out to change his sport. He's done so, raising swimming's profile in America with displays of individual excellence as well as thrilling duels with the likes of Ryan Lochte and Milorad Cavic.
The US Olympic trials are one testament to his success, drawing upwards of 14,000 fans per session. But his more lasting legacy is found in the swimmers he inspired -- some of whom, like le Clos, he will race in Rio and some of whom have yet to reach the international stage.
Once the most intimidating man in the ready-room, Phelps now says he wants to mentor younger swimmers. For the first time he has been selected by his peers to serve as a captain of the US men's team.
"I just want to be able to help some of the younger guys," he said. With swimming heading "in a direction I love," Phelps says there's "still more" for him to do.
"Do I know what that is right now? No. But I still feel that we can do more to promote this sport, even to another level than where we are right now."
But Phelps insists he knows what his future contribution won't include -- another competitive campaign. "This is it. This is it -- no more," he vows. "The body is done. This is my last one."