The reaction followed protests by the Union Public Service Commission (UPSC) aspirants which turned violent Thursday with students burning a bus, a police van and a motor-bike in Mukherjee Nagar area in north Delhi.
Police have arrested 20 people and detained over a dozen.
"The government has full sympathy with the students, and there is also a clarity that under any situation injustice would not be meted out to the students, especially on the basis of a language. We will quickly find a solution to it," Singh told reporters outside parliament here.
Civil services aspirants have been demanding the scrapping of Civil Service Aptitude Test (CSAT), calling it discriminatory against students with humanities subjects and those who had their subjects in Hindi language.
The government has formed a three-member committee to look into the controversy after aspirants continued to protest in several places in the capital.
"We would also like to request and appeal to the students to maintain calm as the government too is worried about the issue. The decisions will be taken on the basis of the committee's report," added Singh.
The protest intensified after UPSC started issuing admit cards to the aspirants for the preliminary exam scheduled to be held August 24.
The minister has also assured that issuing of admit cards does not indicate that it would influence the results of the report by a three-member committee.