The leader of the Yemen-based Al Qaeda branch, identified as Abdullah Hubaibat, was killed when government troops raided a number of houses held by Al Qaeda militants in Loder, a town in Abyan province, the security official said on condition of anonymity.
Five others were also wounded in raids.
Hubaibat was believed to have been involved in previous armed attacks against security forces in Abyan, according to the source.
In the last two months, Yemeni government forces launched anti-terror offensives and drove out scores of gunmen linked to the Al Qaeda and the Yemen-based affiliate of the Islamic State from key neighbourhoods and government compounds in Lahj and Abyan provinces.
Yemen, an impoverished Arab country, has been gripped by one of the most active regional Al Qaeda insurgencies in the Middle East.
The Yemen-based Al Qaeda in the Arabian Peninsula (AQAP), also known locally as Ansar al-Sharia, emerged in January 2009 and claimed responsibility for a number of terrorist attacks against Yemen's army and government institutions.