Local media said the man who died was rushed to hospital after being pulled unconscious from the water by a rescue boat. He was reported to be in his forties.
A woman thought to be in her 60s was separately pulled unconscious from the water and is reported to be in intensive care in hospital.
The 1,500-metre race saw around 3,000 people swim between two piers on opposite sides of Hong Kong's famous harbour -- 500 up from the previous year according to reports.
Some local media questioned why only 10 extra lifeguards had been added when the field had expanded so much. There was a total of 120 lifeguards at the event, broadcaster RTHK reported on its website.
Swimmers are split into racing and recreational groups -- both the victims were taking part in the leisure category, which is for slower swimmers, according to reports.
The Hong Kong Amateur Swimming Association, who organised the event, and the title sponsor New World Development issued a statement expressing their ‘deepest sorrow’ over what they called a tragic accident.
‘The swimmer was rescued but attempts to resuscitate him failed and he subsequently passed away,’ the statement said.
It is the first death in the race since it was resumed in 2011 after a 30-year break, local media reported.
The decades-long hiatus was due to fears over pollution levels in the water.
American swimmer Charles Peterson won the men's title in just 16 minutes 44 seconds this year.
Rio Olympics 10-kilometre open water gold medallist Sharon van Rouwendaal from the Netherlands took the women's crown.