Nazir Ahmed's office in the House of Lords, the car park and the terrace were closed for an hour and a half after he found the powder in a letter containing racist abuse.
"They were treating it as a chemical or biological incident," the peer told AFP, adding: "Thank God it was not dangerous.
"I have received racist, abusive, insulting and Islamaphobic letters before, but never with white powder.
"This was frightening, as our colleague Jo Cox was murdered a few weeks ago."
The letter had been screened before it reached his office, but lawmakers are on heightened alert after Labour Party MP Cox was gunned down on the street in her constituency last month.
A local man, 52-year-old Thomas Mair, has been charged with her murder in the northern English village of Birstall, one week before the EU referendum.
Britain's shock vote to leave the European Union on June 23 came after a divisive campaign focused on immigration, and was followed by a surge in racist incidents.
Ahmed thanked the police and parliamentary authorities for their swift action.
The Metropolitan Police confirmed that officers were alerted to reports of a suspect package at 12:30 pm. The security alert ended around 2:00 pm.
"It was a suspicious package that had been delivered the Houses of Parliament. There are no reports of any injuries or illnesses," a spokeswoman told AFP.
A spokesman for the House of Lords added: "Peers' car park and parts of the terrace were closed temporarily but have now been reopened.
"The closure was put in place while a package containing a white powder was investigated by specialist police units, which is standard procedure.
"The powder was found to be non-harmful."
