The 61-year-old Italian announced the decision a day after Japan were eliminated from the competition, saying he wanted to take full responsibility for the team's failure, reports Xinhua.
"I really wanted to take us forward and qualify for the second round and I am so disappointed. But I picked the team, decided on the tactics and how we play and I want to take full responsibility. I have to leave the Japan team," Zaccheroni told reporters in Brazil.
Appointed as Japanese national team coach in 2010, Zaccheroni's four-year contract was anyway going to expire after the World Cup. Japan now need to rope in a new coach quickly before their Asian Cup defence in January.
Under Zaccheroni, Japan won the Asian Cup in 2011, but the team finished last in Group C in the World Cup group stage in Brazil this time, with one point from a 0-0 draw with 10-man Greece. They lost 1-2 to the Cote d'Ivoire in their opening game and were routed 1-4 in their last game against Colombia Tuesday.
Japan Football Association technical director Hiromi Hara said discussions would take place at a board meeting regarding a successor to Zaccheroni after returning to Japan.