The remarks came while hundreds of furious Australian Muslims launched protest claiming they were terrorised by the operation, Xinhua reported.
"They (the raids) were about (fighting against) potential terrorism and they were about keeping our whole community safe," Abbott told local media.
"It is a safe country but we have to be aware that there are people even here in Australia that would do us harm," said the prime minister in an interview.
The raids sparked wide fury in the Muslim community in Sydney and around 200 residents of the affected area staged a protest at the Lakemba station Thursday night, accusing the police of terrorising Muslims in the country.
Refferring to the protest, Abbott said he "certainly" does not isolate any community and "we are all in this together", adding that the demonstration was small and did not reflect the wider community view.