MPs are put off contributing in the House of Commons by the "histrionics and cacophony of noise" during Prime Minister's Questions, according to John Bercow.
The Commons Speaker said the party leaders had yet to make a specific commitment to improve the situation despite recognising that the behaviour must improve to impress the public.
Mr Bercow said Parliament was "spray painting its own shop window" by appearing to generate higher decibel levels than Deep Purple, regarded as the loudest band in the world in some quarters.
He also claimed there are " down-market parts of the media" who would "positively relish" it if there was a fight on the floor of the Commons chamber.
BBC Radio 4's PM programme said Mr Bercow told them that "seasoned parliamentarians" boycotted PMQs out of embarrassment while some female MPs did not want to take part in the session.
Mr Bercow said: "Not just people standing down but people with a lot to contribute, reluctant to engage in the chamber because they think that the histrionics and cacophony of noise are so damaging as to cause them to look elsewhere.
"But I'm sorry if some of those people are lost to the chamber because they think, 'I'm not going to take part in that atmosphere'.
Mr Bercow was asked i f he had heard back from the party leaders after he sent them a letter about the conduct of MPs in the Commons.
The Speaker replied: "I have heard back from the party leaders.
"There is a general sense, 'Yes Mr Speaker you make a good point and of course we must behave well and try to impress the public and give serious consideration to what people think', but there's not yet much by way of a specific commitment."
Mr Bercow went on: "I know there are people in the Westminster beltway, including in the press gallery, who think, 'Well, what's the Speaker moaning about? Why is he so neurotic? This is the way people like it'.
"To which my answer is no, that's the way you like it.
"There are people in the media, I don't say people necessarily representatives of the PM programme on BBC Radio 4, but there are people in some of the more down-market parts of the media who would positively relish it if there were a physical fight on the floor of the chamber."
In a wide-ranging interview, Mr Bercow was asked about expenses, in the wake of former Culture Secretary Maria Miller's case, and MPs' behaviour, following the not guilty verdicts in the trial of former deputy speaker Nigel Evans.
Mr Bercow said: "If we've changed we have to find ways of explaining how things are different now from what they were.
"And if parliamentarians where they do still have a say over their own colleagues' conduct, reach a different view from say some independent observer or judge, well then we've got a duty to explain perhaps rather better, perhaps more simply, perhaps earlier in the media reporting of a row why we've taken a different view.
"I think in recent weeks we neglected that duty and a story against Parliament was out there before we'd even begun to fight back."
A Channel 4 News investigation claimed last week there was a prevailing "climate of sexual harassment" in Parliament.
Channel 4 News said that its investigation - which involved interviewing 70 people from "all political parties and sexual orientations" - had found sexual harassment and abuse of power in Parliament was commonplace.
Mr Bercow said it was up to Parliament to put in place measures to help staff who suffer unfairly, including a confidential helpline.
He told the BBC: "I think each of the political parties, and I've spoken to the party whips from each of the political parties about this, have got to recognise that it's no good saying the relationship between the MP and the staff is sacrosanct.
"The party's reputation and Parliament's reputation suffers if an MP treats a staffer badly and gets away scot free.
"So the parties are now saying yes, we accept your point, we've got to strengthen our processes.
"Those things need to change."
Mr Bercow also said he was not alarmed by the idea of electronic voting in Parliament and said he would like to hear more views on it.
Told listeners might question why it would be controversial, the Speaker said: "That's a very polite way on your part of saying that we are somewhat behind the times and things that seem to me to be quite innovative would strike a lot of people as being absolutely to be taken for granted."