Kerry's landmark trip is seen as possibly paving the way for Barack Obama to become the first serving US president to visit the thriving metropolis next month, when he comes to Japan for a Group of Seven summit.
The meeting which started Sunday also includes top diplomats from nuclear-armed Britain and France, as well as Canada, Germany, Italy, host Japan and the European Union.
It is part of the run-up to the G7's rotating annual summit, scheduled this year from May 26-27 in another part of Japan.
The ministers were discussing issues including the Middle East, the refugee crisis, the conflict in Ukraine and global terrorism.
Japan also hopes to highlight rising territorial tensions in the South China Sea, where Beijing and some Southeast Asian nations have clashed, and North Korea's nuclear sabre-rattling.
Ministers have so far said little about the content of the meetings, though Kerry tweeted that they had a "big foreign policy agenda to cover" -- mentioning topics such as the Islamic State group as well as "Asia regional issues and global threats".
But it is the location of the talks that has captured the imagination of the Japanese public. Many hope it will promote greater understanding of Japan's staunch anti-nuclear stance as the only country to suffer atomic attack.
Speculation has been building that Kerry's planned visit to a Hiroshima blast memorial on Monday could see the US secretary of state issue an unprecedented apology.
But a State Department official said that was not on the cards.
"If you are asking whether the secretary of state came to Hiroshima to apologise, the answer is no," the official, who asked not to be named, told reporters travelling with Kerry.
"If you are asking whether the secretary and I think all Americans and all Japanese are filled with sorrow at the tragedies that befell so many of our countrymen, the answer is yes."
Japanese foreign minister Fumio Kishida, who represents Hiroshima in parliament, hopes to issue a "Hiroshima Declaration" at the meeting to promote nuclear disarmament.
Kerry and his counterparts are on Monday scheduled to visit the Hiroshima Peace Memorial Park, which houses the ruins of the iconic domed building gutted by the blast, and an accompanying museum.
On Sunday they visited one of Japan's most famous sites, the seaside Itsukushima shrine that dates to the late sixth century.
Shared history
The first American bomb on August 6, 1945, killed 140,000 people in Hiroshima, including survivors of the explosion who died afterwards from severe radiation exposure. Three days later another blast killed some 74,000 people in Nagasaki.
Japan gave up the fight six days after Nagasaki, forswearing militarism and reinventing itself as an economic dynamo -- protected, ironically, by the nuclear-armed United States.
When asked about its place under Washington's nuclear umbrella, Kishida said ahead of the meeting that Japan was mindful of global security realities, citing North Korea as a key threat.
Washington hopes to use Kerry's visit to highlight Obama's anti-nuclear stance.
"Progress on nuclear disarmament must be made in a way that reduces nuclear and security risks for ourselves, our allies and all humankind," Kerry told a local newspaper.
American tourist James Huddleston, visiting the memorial park with his wife and three children, said the top American diplomat's visit was good for both the US and Japan.
"It's important," said the 36-year-old sales manager from Detroit.
"(This) is a big part of the history of both countries."
Hiroshima resident Tatsumi Yamasaki, 78, said a visit by Obama would speak volumes.
"That act itself would mean a lot for people in Hiroshima," he said.
However, a small group of protesters gathered in front of the atomic dome on Sunday to condemn the G7's attitude and any Obama visit.
"They came all the way to Hiroshima to say they would get rid of nuclear weapons -- it's all lies," said organiser Kyoko Taniguchi.
