Police raided a house where the militants were hiding out in Pipri district, some 50 kilometres north east of central Karachi, triggering a gun battle which lasted more than two hours.
Two policemen were wounded in the exchange, officials said.
Rao Anwar, a senior police official who led the raid told reporters that the militants belonged to Al-Qaeda in the Indian Subcontinent and the anti-Shia Laskhar-e-Jhangvi (LeJ) group.
"(They) were involved in target killings and attacks on police," Anwar said.
Police recovered explosive material, bomb making devices, assault rifles and suicide vests, he added.
Eight of the militants were killed during the gun battle inside the house, while four who escaped were later killed in the Gadap area, 12 kilometres southwest of the first encounter.
"Police followed them with the technical support of the intelligence agencies and took them down at Gadap," Anwar said.
Karachi, a city of 20 million and Pakistan's economic hub, is frequently hit by religious, political and ethnic violence. Paramilitary forces began a sweeping crackdown on militants in the city in 2013, which has led to a substantial drop in overall levels of violence.
