The opposition Labour lawmaker, the son of a Pakistani bus driver, broke from convention by taking his oath of office in a multi-faith ceremony at Southwark Cathedral.
"My name is Sadiq Khan and I'm the mayor of London," the 45-year-old said to cheers from supporters, who had earlier given him a standing ovation as he walked in.
He added: "I'm determined to lead the most transparent, engaged and accessible administration London has ever seen, and to represent every single community, and every single part of our city, as mayor for all Londoners."
Khan won 57% of the vote in Thursday's mayoral election, securing 1.3mn votes to see off multimillionaire Conservative Zac Goldsmith and make history as the city's first Muslim mayor.
In his victory speech in the early hours of Saturday morning, Khan had referenced the negative campaign against him by saying London had chosen "unity over division".
Conservative Prime Minister David Cameron had led the attacks against Khan for sharing platforms with radical Muslims at public events, and Goldsmith said he was "radical and divisive".
There was criticism from across the political spectrum on Saturday at the tone of the Tory campaign, but Defence Secretary Michael Fallon insisted it was legitimate.
"Both candidates were asked questions about their backgrounds, their personalities, their judgment, the people they associate with," he told BBC radio.
"That's the nature of our democracy and the rough-and-tumble of politics."
News of the win was applauded in Pakistan, with Bilawal Bhutto, leader of the opposition Pakistan People's Party and son of former prime minister Benazir Bhutto, and rival opposition leader Imran Khan tweeting congratulations.
New York Mayor Bill de Blasio said he was looking to working with his "fellow affordable-housing advocate" while Paris Mayor Anne Hidalgo tweeted that Khan's "humanity (and) progressivism will benefit Londoners".
Former Conservative government minister Sayeeda Warsi also offered her congratulations "from this daughter of a Pakistani bus driver to the son of a Pakistani bus driver", and condemned her party's campaign.
"Our appalling dog-whistle campaign lost us the election, our reputation and credibility on issues of race and religion," she said.
Khan admitted representing some "pretty unsavoury characters" during his previous job as a human rights lawyer but said their views were "abhorrent" and condemned the Conservatives' "desperate" attacks.
Goldsmith's sister Jemima, the ex-wife of Pakistani cricketer and politician Imran Khan, said the tone of her brother's campaign "did not reflect who I know him to be".
Cameron's former adviser, Steve Hilton, said Goldsmith had brought back the "nasty party label".
In the audience at Southwark Cathedral was Doreen Lawrence, an anti-racism campaigner whose teenage son Stephen was killed by a gang of white youths.
"I never imagined in my lifetime I could have a mayor of London from an ethnic minority," she said.
Labour losses elsewhere
Khan has broken the eight-year hold of the Conservatives on City Hall, succeeding the charismatic Boris Johnson in a prestigious post that has responsibility for transport, housing, policing and promoting economic development.
His success was a boost for Labour leader Jeremy Corbyn, a veteran socialist who has been battling a row over anti-Semitism and growing criticism from the moderate wing of his party since his election in September.
But Labour fared less well in other regional elections on Thursday.
The party was beaten into third place in Scotland, once a Labour stronghold, as the Conservatives became the official opposition to the Scottish National Party (SNP), which won a third term in office.
Labour maintained control of the Welsh assembly and lost only a handful of local council seats in England.
But critics warned it should have done better against a government that has lost support over welfare reforms and is deeply divided ahead of the referendum on Britain's membership of the EU on June 23.