A fine Keisuke Honda strike after 16 minutes had put Japan in front at the Pernambuco Arena in Recife, but the Ivorians introduced Didier Drogba from the bench just after the hour mark before going on to turn the game on its head.
Within five minutes of the veteran striker's arrival on the rain-soaked surface, the Elephants scored twice, both times from Serge Aurier crosses. The equaliser came when the right-back's curling delivery was headed home by Wilfried Bony, before Gervinho connected with another Aurier centre to stun the sizeable Japanese contingent inside the stadium.
The result sees the Ivory Coast join Colombia -- who beat Greece 3-0 earlier in the day in Belo Horizonte -- on three points at the top of Group C, and Sabri Lamouchi's side must face the South Americans next in Brasilia on Thursday, with Japan next taking on Greece in Natal.
The Elephants are desperate to finally progress beyond the group stage at a World Cup after failing in 2006 and 2010, and they were boosted in Recife by the presence in their starting line-up of Yaya Toure, who shrugged off a hamstring problem in time to captain the side.
Drogba was left on the bench at kick-off, with Swansea City's Bony preferred as the spearhead of the attack, but the Ivorians still maintained a considerable physical advantage over the Asian champions.
However, what Japan lack in muscle they more than make up for in technical ability, and Honda displayed his class to put the Blue Samurai ahead. Following a throw-in from the left flank, the ball arrived at the feet of AC Milan playmaker Honda at the edge of the box, and he took a touch before letting fly with a powerful, rising left-foot strike that gave Boubacar Barry Copa in goal no chance.