Deputy Commons Speaker Nigel Evans has denied accusations of raping and assaulting two young men saying the allegations were "completely false".
The 55-year-old Tory MP described his arrest on Saturday at his home in the village of Pendleton, Lancashire, and subsequent questioning by police as "the worst 24 hours of my life".
He made clear through his solicitor that he intended to carry on as Deputy Speaker while police continued their investigation.
Mitchell has spoken out about the rape allegations against Evans
Speaker John Bercow has however agreed that he should be excused his duties chairing the Queen's Speech debate which opens on Wednesday and continues to Wednesday the following week.
Following his release on Saturday night on bail, Mr Evans emerged yesterday morning in the beer garden of the village pub next to his cottage to deliver a brief statement expressing his "incredulity" at what had happened.
Looking tired and tense he said: "I was interviewed by the police concerning two complaints, one of which dates back four years, made by two people who are well known to each other and until Saturday, I regarded as friends.
"The complaints are completely false and I cannot understand why they have been made, especially as I have continued to socialise with one as recently as last week.
"I appreciate the way the police have handled this in such a sensitive manner and I would like to thank my colleagues, friends and members of the public who have expressed their support and, like me, a sense of incredulity at these events."
Lancashire Police said Mr Evans, who has been MP for the Ribble Valley constituency since 1992, had been arrested on suspicion of raping one man and sexually assaulting another between July 2009 and March 2013.
Following several hours of questioning by detectives at Preston Police Station, he was released on bail to June 19.
Defence Secretary Philip Hammond questioned whether he could continue as Deputy Speaker while he was under investigation.
"I stick rigidly to the view that we should treat people as innocent until they are proven guilty but it is quite difficult to carry out a sensitive and high profile role while being under this kind of scrutiny," he told BBC1's The Andrew Marr Show.
Overwhelmingly however the mood both among MPs and among residents in the village appeared strongly supportive of Mr Evans, who has long been a popular figure at Westminster.
Foreign Secretary William Hague described him as a "long-standing friend" and said that MPs of all parties would be "very sorry to see this situation".
Tory MP Andrew Bridgen, who was accused of a sex assault two years ago only for police to drop their inquiries six days later, also voiced his support .
"Fortunately in this country we have a rule that says you are innocent until proven guilty and I think that should be maintained," Mr Bridgen told Sky News.
"I personally think that Nigel should be able to continue as Deputy Speaker while the police, quite rightly, carry on with their investigations."
A statement on the Ribble Valley Conservatives Association website said Mr Evans was "widely liked and respected" and they expected him to continue with his duties in the constituency as normal.
"In our democracy everyone accused is innocent until proven guilty and therefore unless Nigel chooses himself to cease to be our MP or the electorate vote him out or justice system intervenes, we expect him to continue as normal to fulfil his duties in representing the people of the Ribble Valley," it said.
One Pendleton resident, who did not want to give his name, said: "It's unbelievable, I know him very well. He is an excellent person and I would stand by him 100%."
Mr Evans later made a second brief appearance before the cameras as he returned to the pub for his "Sunday pint" to express his gratitude for all the support he had received.
"I am so grateful and it's that support that is really helping me get through this. I would just like to thank, from the bottom of my heart, everyone who has sent me a message. Thank-you very much," it said.
Mr Evans was one of three deputy speakers elected in a secret ballot of MPs in 2010. Later that year he came out as gay, saying he was "tired of living a lie".
He told The Mail on Sunday that he had also been threatened with exposure by political opponents.
"I could not afford it to be used as leverage against me. I couldn't take the risk. I don't want any other MP to face that kind of nastiness again," he told the paper.
"I am sure there are other gay MPs who would like to be open about their sexuality but are fearful of the consequences."
Lancashire Police said in a statement: "We take all allegations of a sexual nature extremely seriously and understand how difficult it can be for victims to have the confidence to come forward.
"As a constabulary, we are committed to investigating sexual offences sensitively but robustly recognising the impact that these types of crimes have on victims."
Andrew Mitchell, who quit the Cabinet amid a media storm over claims - which he strongly denies - that he called police officers 'plebs', said Mr Evans should remain in his post.
"If he were to resign now we would be in a terrible position where every public office holder would be open to blackmail and false stories in the media, triggering their resignation," he told BBC Radio 4's Today.
"Nigel Evans has been a friend and colleague of mine for more than 20 years, I have known him in good times and in bad times and I certainly do not believe these allegations.
"I think he should definitely continue in that role. We have three deputy speakers in the House of Commons so if necessary there can be a degree of burden- sharing.
"He has not been charged, he has not been found guilty, and we do still live in a country where you are supposed to be innocent until proven guilty."
Former chief whip Mr Mitchell strongly denies branding officers "plebs" when they stopped him cycling out of the main Downing Street gates last September, but has admitted swearing.
He was eventually forced to resign over the controversy, but doubt has since been cast on some accounts of his behaviour.
Scotland Yard is investigating claims of a conspiracy by officers to unseat the former Cabinet minister and his lawyers have begun libel proceedings against The Sun.
The newspaper, which broke the story, says it will vigorously contest the action.
Asked why he resigned, Mr Mitchell said: "In my circumstances, this matter had dragged on for 28 days. I took the view that I could no longer discharge the duties the Prime Minister had asked me to discharge and that's why I resigned.
"Anyone who is caught up in the sort of intensive media storm which Nigel has faced and which I and my family faced, obviously is in a very difficult position to carry out their job, But in this particular circumstance I am in no doubt that Nigel Evans should not resign and that he will be able to fulfil his duties as required by the Speaker."