Congress spokesperson and union minister Shashi Tharoor said the statement made by Priyanka Gandhi had "no bearing" on Modi's caste.
He said the statement was made in reference to the "debased level of political discourse" on the part of the Bharatiya Janata Party and its prime ministerial candidate.
He said the statement was made by Priyanka Gandhi after Modi made a speech in Amethi that contained allegations "unacceptable to her".
Tharoor added that Modi "tried to exploit that phrase and attach the idea of caste to it".
The Congress leader said Modi's statement meant he was asking the electorate to vote for its caste and that it "revealed how he thinks about caste".