Ben de Pear said the decision was taken after six men, who claimed to be from immigration police, came to his hotel room on Sunday demanding entry.
The men accused the journalists of breaching their visa conditions by "attempting to enter the president's palace".
The news team has always been a target of Pro-government protesters for the channel's award-winning documentary "No Fire Zone: Sri Lanka Killing Fields".
"People often get visits in this country and are bundled off in white vans never to be seen again," Ben told AFP.
The summit ended on Sunday. The whole event was overshadowed by row over alleged war crimes by the Sri Lankan army during the infamous Eelam war that ended in 2009. International community has been constantly pushing for justice to those who were tortured and killed during the war.
British Prime Minister David Cameron demanded an international inquiry into the war crime allegations.
During its stint in Sri Lanka, the UK-based news team was constantly monitored and was stopped from visiting a former war zone.
(With agency inputs)