Michael Gove has insisted that his department is not a "gallery of nodding dogs" amid a warning that he should not surround himself with "yes men".
Civil servants at the Department for Education (DfE) are skilled at saying "yes, no, maybe" to ministers, the Education Secretary said.
And as the row over his decision to remove Baroness Sally Morgan of Huyton from her post as Ofsted chair continued, Mr Gove insisted that the Labour peer's replacement will be whoever is best for the job, be they a "member of a revolutionary communist party" or a political party donor.
His comments came after Sir David Bell, vice-chancellor of Reading University, warned the minister in an online blog not to surround himself with "yes men".
Sir David - who previously served alongside Mr Gove as the senior civil servant at the DfE - warned that the minister should "not believe his own hype" and allow daily political life to cut him off from outside ideas and thinking.
Asked about the comments during a visit to the London Academy of Excellence in east London today, Mr Gove said: "I don't think that's true. One of the best things about the DfE is that it attracts brilliant civil servants who are skilled at saying yes, no, maybe or just sucking their teeth if they want to let me know it's a bad idea."
He added: "The characterisation of the DfE as a gallery of nodding dogs is very wide of the mark."
Writing for the academic website The Conversation, Sir David - who is also a former Ofsted chief inspector - warned Mr Gove that he risks becoming isolated by listening only to his own supporters.
He said: "The day-to-day grind of policy battles, firefighting and political ding-dong can start to cut you off from outside ideas and thinking.
"The row over Ofsted's leadership shows the importance of retaining and being seen to retain independent voices near the top - not simply 'yes men'."
Mr Gove became embroiled in a row over the weekend after it was revealed that he has decided not to reappoint Labour peer Lady Morgan for a second three-year term as Ofsted chair.
He paid a glowing tribute to her ''enormous contribution'' to the work of Ofsted, insisting that the move was to "refresh" the watchdog's leadership and denying claims that it was politically motivated.
But Mr Gove faced accusations by his Lib Dem colleagues of trying to "politicise" the schools inspectorate.
The decision was seized on as a fresh attempt to assert his influence over the regulator, with Mr Gove's normally loyal Lib Dem deputy David Laws accusing him of trying to pack the watchdog with his ''own people''.
Mr Gove today insisted that there is an independent system in place for public appointments.
An independent body oversees the appointment process, with their recommendation sent to the relevant Secretary of State to approve or veto.
Mr Gove said "By definition, that independent body will make its recommendation entirely free of any consideration about the political views or background of any candidate.
"So if the right candidate, for any public appointment, happens to be a member of the revolutionary communist party or someone generous enough to support a political party with their hard earned cash, if they are the right person then he or she will be appointed and that's the end of it."
In his blog Sir David also had strong words for the main teaching unions, saying that their leaderships had "played right into the Government's hands over the past four years.
"Their barrage of industrial action and knee-jerk opposition to any change, has allowed the Education Secretary and his supporters to characterise them as cartoon-like bogeymen," he wrote.
"The unions' political naivety has been astonishing."
The row over Lady Morgan's departure comes just a week after Chief Inspector of Education Sir Michael Wilshaw voiced his anger at the Education Secretary in the wake of media reports that two right-wing think-tanks were drawing up plans to reform or even replace Ofsted.
Mr Gove was forced to issue a statement voicing his full support for Sir Michael and promising to dismiss any aides if they were found to be part of any ''dirty tricks'' campaign.
Mr Gove told BBC Radio 4's The World At One that Baroness Morgan's successor would be appointed after a "rigorous" independent process.
Asked if Mr Laws would be responsible for finding a replacement, he replied: "No. No politician will have responsibility for finding a replacement.
"That will be done by an independent group of individuals who, under the rules that have been set in place since John Major's time, have the responsibility for interviewing candidates and then they will make a recommendation about who is, or is not, appointable and then it is the relevant Secretary of State who makes the final decision whether or not to accept or to veto that independent recommendation."