There were separate serpentine queues of men and women voters in areas like Peeli Kothi and Beniyabagh where most people are Muslims, officials and political activists said.
The huge numbers of voters included women too, many in burqa. There were also the elderly and some on wheel chairs.
The huge Muslim turnout was expected after Muslim clerics came out in support of Aam Aadmi Party leader and candidate Arvind Kejriwal.
The Mufti Board here called upon Muslims Sunday to vote on the AAP's "broom" symbol, saying Kejriwal was a "honest man".
Muslims form a large chunk of the Varanasi electorate, and it was earlier believed that the community's votes may split between Kejriwal and Ajay Rai of the Congress.
The polling percentage in Varanasi had crossed 43 percent by 3 p.m., an official told IANS.
"This appears to be a good turnout. We are sure the voting will cross all past records," the official said.
Voting percentage in the 2009 Lok Sabha polls was 46 percent.