Lord Greaves said he was concerned EU nationals could face treatment "reminiscent" of 1930's Germany as he urged the Government to do more to combat "hostility" after the UK’s exit from the bloc.
The controversial comments came as peers debated amendments to Boris Johnson's Withdrawal Bill, which seeks to cement 31 January as the UK's official exit date from the European Union.
Speaking in the upper chamber he said: "I am fearful on the 31st January that some things may happen in some places which could be reminiscent of things happening in Germany in the early 1930s.
"I am worried because there is that sentiment among a hostility minority of the population and I'd like to know what the government is trying to stop this happening."
Lord Greaves also hit out at "unfeeling hard-headed Tories" as he claimed the Brexit debate had caused "very serious wounds" in the country.
"People are moaning, people are crying at night when they go to sleep, they are crying in the morning when they wake up," he added.
"And all they get from the unfeeling hard headed Tories is moans. They are feeling a sense of loss that is akin to bereavement and a grieving process has only just begun."
But Labour peer Lord Grocott said the comments left him "reeling", adding: "He's just made a comparison...between Britain on Febraury 1st this year, and Nazi Germany in 1933....that seems to be stretching the point just a bit."
Lord Greaves then defended his remarks, saying some European citizens were now afraid to go into shops because of their accents.
"The day after the referendum, people had their windows put in, people were abused in the street, paint was daubed on people's houses, that kind of thing," he said.
"All I know is that talking to EU citizens here, I know people who are reluctant to go into shops now if they are not known in those ships because of their accent, and the attitude that people might have towards them."
And he said the behaviour was "widespread but not very frequent", adding: "I am very worried that on the 1st February and 2nd February there will be a wave of this kind of thing."