DISABLED people should stop relying on the state and “work their way out of poverty”, according to Iain Duncan Smith.

Addressing the Conservative party conference in Manchester, the Work and Pensions Secretary said it should not be up to the Government to stop the disabled being poor.

Duncan Smith said: “We don’t think of people not in work as victims to be sustained on Government handouts. No, we want to help them live lives independent of the state.”

The minister claimed that most sick and disabled people wanted to work. It was up to the Government, he said, to help them back into work.

“We won’t lift you out of poverty by simply transferring taxpayers’ money to you," he said. "With our help, you’ll work your way out of poverty.”

Thousands of disability campaigners staged a noisy protest outside the conference centre during Duncan Smith’s speech. It followed two days of protest that, while mostly good-natured, has seen delegates being called "scum" as they walk into the conference.

The Tory said his party should not be intimidated by the protests: “We watch outside and we see that vile abuse and the left ranting and screaming at us and telling us all sorts of things. We stand up to them and we will not be moved. It is our country that we care for and our countrymen and women.”

Duncan Smith also criticised Labour and the old Employment Support Allowance. He said the benefit, for people too sick to work, made claimants “passive”.

“The ESA has Labour’s essential mistake at its heart – that people are passive victims." he said. "Of course, if you treat people as passive that’s what they’ll become. It’s no wonder, when the system makes doctors ask a simplistic question: Are you too sick to work at all? If the answer is yes, they’re signed off work – perhaps forever.”

He continued: “So we look to change the system – and the assumptions that underpin it. Conservatives' philosophy is rooted in human nature – not in Utopianism or in empty pity but in the yearning of people to make a better life for themselves and their children.

“That’s why we don’t think of people not in work as victims to be sustained on Government handouts. No, we want to help them live lives independent of the state. The evidence of our reforms is that people respond to incentives.”

Charity Scope warned the Work and Pensions Secretary that his cuts would undermine any ambition he had to get people back into work.

Mark Atkinson, the charity’s director said: “The Secretary of State is right to say that many disabled people can, and want to, work. If we are going to halve the disability employment gap, we need to remove the barriers disabled people still face when finding, staying in and progressing in work.

“However, we are deeply concerned that lowering the financial support unemployed disabled people receive will push people further from the workplace. Those in the work-related activity group who receive Employment Support Allowance are disabled people who’ve been independently assessed as being unfit for work. It is not a passive benefit. Everyone in the group must take steps towards finding work.”