Narendra Babu ended with 40 votes.
Gowda later said in his Facebook account: "Folks, thanks for all your good wishes ... but the results were quite different from what I had worked for. There's always a next time."
Narayanaswamy, who had served as a Janata Dal MP from Bangalore North constituency between 1996-98 and known for his integrity, will take on the BJP's Sadananda Gowda, a former chief minister, from the seat after the Central Election Committee gives its nod. Narayanaswamy had won in 1996 against the Congress's Mohammad Obaidulla Sharief by a huge margin of over 1.3 lakh votes.
Narayanaswamy joined the Congress after quitting the JD(S) in August last year.
Rajeev Gowda, a professor at IIM-Bangalore, is known for his initiatives for women's empowerment, development for the youth and improvement in civic development.
The defeat of Gowda, whose symbol in the primary election was table, had left many of his supporters shocked and disappointed.
The only other seat in the southern for which a primary was held by the Congress is Dakshina Kannada. There, former KPCC chief Janardhan Poojary emerged winner.