The Interfax news agency, quoting a police source, said "a religious radical and probably a follower of non-traditional Islam", was on the rampage in central Almaty, opening fire from time to time.
Two witnesses told Reuters they had heard shots in several areas in the centre of Almaty, the mainly Muslim country's biggest city.
"We saw a man with a rifle, he passed by," one shop worker said by phone.
Police cordoned off several central streets, including the one near a local office of the KNB security police, where shots were also heard.
"I heard one shot, most probably, fired from a pistol," said one man standing nearby.
At least one attacker had been detained by the police, the RIA news agency quoted a security source as saying. It said several policemen had also been wounded.
Kazakhstan, an oil rich-rich nation of 18 million, is far more prosperous than its post-Soviet neighbours in Central Asia.
President Nursultan Nazarbayev, 76, has ruled it with a firm hand since 1989, making stability his motto.
But the country has recently seen outbreaks of violence, initially triggered by discontent over proposed land reforms.
The KNB, successor to the Soviet-era KGB, said last month it had detained several members of a group which planned "terrorist acts using improvised explosive devices", following a deadly attack in the northwestern town of Aktobe.
In that incident, about two dozen men described by the authorities as sympathisers of Islamic State, attacked gun stores and a national guard facility, killing seven people.
Security forces killed 18 attackers, some on the same day and some in the subsequent manhunt.
A bullet hole is seen on the window of a car parked in the street in Almaty, Kazakhstan
Gunmen killed two policemen in Kazakhstan's financial capital Almaty on Monday, security sources and the Interior Ministry told Reuters, as the authorities said they had launched an anti-terrorist operation.
