In the survey, commissioned by charity Scope 46% reported that people's attitudes towards them have got worse over the past year.
The charity said it is "impossible to ignore" increased hostility in a climate of increased media report about "benefit scroungers".
Disabled people report that they are increasingly confronted by strangers questioning their right to support.
73% experienced the assumption that they don't work; 83% say coverage about benefits scroungers can negatively affect attitudes; 87% say benefit scroungers themselves have a negative effect on attitudes.
Richard Hawkes, chief executive of Scope, said:
"It is telling that these figures come as the Government continues to put the issue of weeding out illegitimate claimants at the heart of its welfare rhetoric.
"The facts and figures they release on welfare reform only tell half the story. Benefit fraud is rare – in fact more money goes unclaimed than is defrauded, and the new fitness for work test is shown to be failing miserably to accurately assess people's likelihood of finding work.
"This backdrop of negativity will only make it harder for disabled people to overcome the many barriers they face when it comes to getting on with their lives."
Scope said only a tiny number of people claiming disability benefits are fraudsters, yet the way their actions are reported is one of the chief causes of public hostility.
"It is absolutely shocking that in 2012 almost half of disabled people feel attitudes have got worse and many have experienced aggression, hostility or name calling from other people," Mr Hawkes said.
"Disabled people keep coming back to the same concern: benefit scroungers. They single out fraudsters. They are concerned about coverage. They tell us strangers challenge them in the street about the support they claim.
"Yet fraudsters are a tiny minority of claimants.
"It is telling that these figures come as the Government continues to put the issue of weeding out illegitimate claimants at the heart of its welfare rhetoric.
"The facts and figures they release on welfare reform only tell half the story. Benefit fraud is rare – in fact more money goes unclaimed than is defrauded, and the new fitness for work test is shown to be failing miserably to accurately assess people's likelihood of finding work."
Mr Hawkes said the Paralympics are "a once-in-an-lifetime opportunity to stop this deterioration and leave a lasting legacy of improved attitudes towards disability".
"Scope will be working throughout the games to tell the stories of disabled people in 2012.
"We want the Government to mark the games with a new approach to welfare: tell the whole story when it comes to stats; make fundamental changes to the Work Capability Assessment and avoid repeating the same mistakes when it comes to the new assessment for Personal Independence Payments."