Navy spokesman Kosala Warnakulasuriya told Xinhua that the fishermen were arrested overnight after they crossed the international maritime boundary line in the northern seas.
A group of 22 fishermen was arrested and their boats seized off Thalaimannar in the north while another group of 15 Indian fishermen was arrested off the Delft Islands, also in the north.
The navy spokesman said the fishermen would be brought shore and handed over to the police for investigations.
The issue of fishermen from both countries illegally crossing the maritime boundary line has been a contentious problem for India and Sri Lanka.
Indian fishermen often accuse the Sri Lankan Navy of attacking them when they inadvertently enter the Sri Lankan waters, a charge the Sri Lankan Navy denies.
India's Foreign Minister Salman Khurshid discussed the issue with Sri Lankan President Mahinda Rajapaksa when he visited Sri Lanka recently.
It was agreed that the best way forward would be to facilitate a dialogue between fishing communities of both countries so that the fishermen themselves can find solutions to their problems.
Recently, many Indian fishermen, who were arrested by the Sri Lankan Navy, were released and sent back to India but many others still remain in detention in the island-country.