Detectives visited his home in Pendleton, Lancashire, on Saturday following allegations by two men who were in their 20s at the time of the alleged attacks, which are said to have taken place at the property in July 2009 and March 2013.
The house was searched by officers and Deputy House of Commons speaker Evans was held in custody at Preston police station. He was subsequently released on bail till June 19.
British Prime Minister David Cameron is understood to have been informed about the arrest of the 55-year-old MP for Ribble Valley, one of the few Tory strongholds in the north.
Commons speaker John Bercow last night said: "It's been a total shock. Nigel is a popular and well-liked character. No one can quite believe what's happened."
Evans is openly gay, having come out as homosexual in 2010, the year he was elected to his current post.
In the 2010 general election Evans retained his seat with a majority of 14,769 over Labour, with Ukip making a then strong showing in fourth place with 3,496, or 6.7 per cent of the vote.
Lancashire police said: "A 55-year-old man from Pendleton in Lancashire has been arrested on Saturday by Lancashire Constabulary on suspicion of rape and sexual assault.
"The man will be interviewed at a police station in Lancashire during the course of the day. The offences are alleged to have been committed in Pendleton between July 2009 and March 2013. We take all allegations of a sexual nature extremely seriously and understand how difficult it can be for victims to have the confidence to come forward.
"As a constabulary, we are committed to investigating sexual offences sensitively but robustly, recognising the impact that these types of crimes have on victims. We would encourage anyone who has experienced sexual abuse, or who has information about it, to have the confidence to report it to us knowing that we will take it seriously, deal with it sensitively and investigate it thoroughly."
Evans, who earns 102,000 pounds a year as an MP and deputy speaker, was first elected as a councillor, while working in his family's newsagent business, to West Glamorgan county council in 1985. Five years later he became leader of the Conservative group.
Evans is known as a rightwinger who voted against the minimum wage and is a strong Eurosceptic.
Originally from Swansea, he was John Major's spokesman on Welsh affairs. He held various positions as a parliamentary private secretary in Major's government including to Tony Baldry in the ministry of agriculture, food and fisheries.
In 1999 he was appointed as vice-chairman of the Conservative party, a post he held until 2001. At the time of his coming out as gay he said he was tired of living a lie and dealing with threats of blackmail from political enemies.