A Tory MP has warned Theresa May that she will struggle to carry on as Prime Minister in the wake of her mishap-strewn party conference speech.

Former minister Ed Vaizey said “quite a few” in the party were now “pretty firmly” of the view that she should stand down.

His warning came as Mrs May’s allies sought to rally round her, insisting that there was no mood in the party for a leadership challenge.

So far critics on the backbenches have been reluctant to voice their concerns publicly.

However Mr Vaizey told BBC Oxford: “I think there will be quite a few people who will now be pretty firmly of the view that she should resign.

“The Tory Party conference was a great opportunity to reboot the party and therefore reboot the country to give it a clear sense of direction and that didn’t happen. So yes, I am concerned. I am finding it increasingly difficult to see a way forward at the moment, and it worries me.”

The already febrile mood in the party was heightened by Mrs May’s conference address in Manchester when a prankster managed to hand her a P45, part of the scenery fell down, and she was beset by a persistent cough.

There was anger among loyalist MPs who accused the rebels of a “cowardly” attempt to circumvent party rules in order to unseat her.

Under the rules, 48 MPs would need to write to the party’s backbench 1922 Committee expressing no confidence in Mrs May in order to trigger a leadership contest.

But loyalist backbencher Mark Pritchard expressed concern that the rebels were trying to put together a delegation to tell her the time had come for her to go.

“If some MPs want to see a change of leader there is an agreed process through the 1922 Committee,” he told the Press Association.

“Trying to drum up a delegation of 30 MPs to try and circumvent this process is irregular, cowardly and will ultimately fail. Any minister with premature ambitions needs to put up or shut up and allow the Prime Minister to get on with her day job.”

Mrs May’s speech had been seen as a chance to reassert her authority after seeing her Commons majority wiped out in the June general election and Cabinet feuding over Brexit.

But she woke up to newspaper headlines describing Wednesday’s events in the Manchester Central hall as a “nightmare” and a “shambles”.

Business Secretary Greg Clark, a close ally of Mrs May, insisted the party was fully behind her and praised the way she carried on with the speech, despite all the difficulties.

Key words in Theresa May's conference speech(PA Graphics)

“A lot of the people who were there in the hall and watching it on TV will have admired, frankly, the guts and the grace the PM showed,” he told BBC Radio 4’s Today programme.

“One of the things that established itself very clearly was that the view of the party – of both parliamentary colleagues and activists – is that they regard, correctly, the responsibility of the Conservative Party to be effective and cool-headed in government.

“They have got no patience for distractions from that.”

Backbencher George Freeman, who heads Mrs May’s policy board, said a leadership challenge risked handing the keys of No 10 to Labour leader Jeremy Corbyn.

Theresa May and Jeremy Corbyn's conferences speeches compared(PA Graphics)

“What we need to do is focus on sharpening up, smell the coffee from the Corbyn threat,” he told the BBC’s Victoria Derbyshire programme. “The Cabinet needs to get a grip and debate the issues of the day in the Cabinet, not in the media.

“Comrade Corbyn and Comrade McDonnell and the new socialist insurrection are waiting for us to fail. We mustn’t fail. If we fail, we will not just let the Conservative Party down, we will let the country down.”

Much of the anger among newer Tory MPs is understood to be directed towards Boris Johnson, who is perceived to have been overtly disloyal to Mrs May by setting out his Brexit red lines on the eve of the conference.

His allies are said to have stepped up efforts to canvass potential support for a leadership bid following the PM’s speech. But there are suggestions he does not have the required support of 48 MPs and that many want him sacked.

If MPs voice strong opposition to the Foreign Secretary continuing in post at next week’s 1992 Committee meeting, its chairman Graham Brady could raise the issue in his regular meetings with the PM, it is believed.

It is understood that some 2015 intake MPs even confronted Mr Johnson face-to-face at the conference in an effort to get him to fall in line.