Though most of the Muslims are not inclined towards Hussain, yet he and his Bharatiya Janata Party (BJP) are eyeing the division of their votes in Bhagalpur, known as silk city, between Nitish Kumar's Janata Dal-United (JD(U)) and Lalu Prasad's Rashtriya Janata Dal (RJD), which seeking to make inroads into the Kurmi votes.
It is a triangular contest between Hussain, RJD's Shailesh Kumar alias Bulo Mandal and JD(U) Abu Qaiser.
Kamal Ahmad and Rahmat Ansari, residents of Nathnagar area here, had voted for Hussain in 2009 due to Nitish Kumar but say will not follow suit now. "We supported Shahnawaz Hussain last time when Nitish Kumar's JD-U was in alliance with BJP. But now, there is no question to supporting him," said Ahmad
Ansari said with Modi the BJP's prime ministerial candidate, Muslims have strong reservations against Hussain. "Voting for Shahnawaz Hussain means supporting Modi, whom by and large Muslim continue to hold responsible for the 2002 Gujarat riots," he said.
However, Kamran Khan says Muslims likely to vote for a candidate who can defeat the BJP but they are divided between support to RJD and JD(U) so far. "Division of Muslims and other anti-BJP votes will help Shahnawaz again," he said.
Abu Qaiser's presence in fray has come as a blessing in disguise for Hussain, said Dr Sambey, who stays in the heart of Bhagalpur town, near the railway station. "BJP and Shahnawaz are hoping for division of Muslim voters because battle is tough this time. Once BJP supporters, Mandal's castemen, locally known as Gangota - an extremely backward caste - who have strong presence here, are openly backing Mandal," he claimed.
It was the full support of Gangotas in 2009 that mainly contributed to Hussain's victory. The situation has changed now, a bad sign for BJP despite so-called Modi wave," said Sambey, who has been watching polls closely here for last over two decades.
Mandal's Gangota castemen have more than three lakh voters here in riverine belt of Gopalpur and Bihpur assembly segments and a sizeable presence in other assembly segments.
Shahnawaz Hussain is looking to a division of votes between the JD(U) & RJD
With caste factor set to play by all three of them to consolidate their support base, but Mandal, a former legislator, has an edge over Hussain, said Sambey. Besides Mandal's own castemen, he is sure of getting overwhelming support of another vocal backward caste - the Yadavs - due to their loyalty to his party chief Lalu Prasad, who is still seen as their undisputed leader, as well as a major chunk of over three lakh Muslim voters.
"Shahnawaz Hussain won the seat in last two polls due to crucial support of Gangota caste. But Mandal is now giving sleepless nights to him," said Sambey.
Kurmi, another backward caste, with strong presence in Kahalgaon,Pirpainty and neighbouring areas, supported Hussain last time thanks to Nitish Kumar.
"If upper castes are supporting BJP, Yadavs are supporting Lalu Prasad, Kurmis are supporting Nitish Kumar. We will vote for the JD-U candidate, not Shahnawaz Hussain," said JD-U leader Suresh Prasad Singh, a Kurmi.
The JD-U is also heavily banking on support of Mahadalits - the poorest among Dalits.
For Hussain, a former union minister, the silver lining is BJP's strong support base in Bhagalpur town and other semi-urban areas, where upper castes along with Baniya or Teli backward caste and others are openly backing him in name of Modi.
The BJP is also banking on their alliance with Lok Janshakti Party of Ram Vilas Paswan to seek support of Dalit Dusadh caste.
However, Hussain himself expresses confidence that the he will retain the seat with a bigger margin this time. "I am sure that the Modi factor will help me," he says.
Last week Modi had addressed an election meeting in Bhagalpur in Hussain's support.