The refugees lodged in Vridhachalam and Tiruchirappalli camps were spotted standing near the seashore off Sonankuppam village in the early hours by the local villagers, who informed police.
During questioning, the refugees told police that they had paid about Rs seven lakh to an agent for taking them to Australia by a mechanised boat, police said.
The state Q branch police registered a case and are searching for the agent hailing from Vridhachalam in Cuddalore district.
Since last year, there had been several instances of Sri Lankan refugees attempting to illegally travel to Australia and other countries from different coastal towns in Tamil Nadu, neighbouring Puducherry and Kerala seeking greener pastures.
Last week, a group of nine Sri Lankan nationals were detained in Kochi following suspicion that they were trying to travel illegally to another country.