Some 100 protesters rallied outside the Vietnamese Embassy, which was fortified by anti-riot police and heavy steel barricades, Xinhua reported.
Some of the activists trampled on Vietnamese flags before setting them on fire outside the embassy.
"The purpose of the protest remains the same. We're demanding an apology from the Vietnamese diplomat and urging him to accept the 'true history' of the former Kampuchea Krom provinces," said Thach Sitha, president of the Khmer Kampuchea Krom Association.
The group has planned to stage a five-day demonstration despite no permission from the Phnom Penh municipality.
The minority Khmer Krom community has staged several protests against Vietnam in recent months after Trung Van Thong, a spokesman for the Vietnamese embassy to Cambodia, commented on a radio programme in June that South Vietnam, which protesters claim was once part of former Kampuchea Krom provinces, belonged to Vietnam long before France's official transfer of the land in 1949.
During a protest in August, demonstrators set a Vietnamese flag on fire in front of the Vietnamese Embassy and triggered a strong reaction from the Vietnamese government.
No one has been arrested after the incident.