While Grabar-Kitarovic of the Croatian Democratic Union got 50.5 per cent of the ballots, her opponent got 49.5 per cent. Ninety-nine per cent of the ballots were counted, a BBC report said.
Josipovic, who was in office since 2010, conceded defeat and congratulated the opponent.
The turnout in the second round of polling was 58.9%, which is about 12% more than in the first round held two weeks ago.
The conservative challenger's victory could indicate a shift in Croatia's move towards the right after the ruling centre-left coalition's failure to end six years of economic under-performance. Croatia, which joined the European Union in July 2013, has an unemployment rate near about 20 per cent.
The latest election is a key test for the country's main political parties ahead of the parliamentary election due to be held towards the end of this year.
The Croatian Democratic Union played a major role in pushing the country towards independence from former Yugoslavia in 1991 and Grabar-Kitarovic, 46, is a former foreign minister and assistant to the Nato secretary general Jens Stoltenberg.