The final operation to save the men trapped more than 200 metres (660 feet) underground took two hours as they were hauled up to the surface one by one in a rescue "capsule", according to state broadcaster CCTV.
The four were among 29 trapped when the mine collapsed on December 25. But the rescue was marred as a local government official said there had been no contact with 13 other missing miners.
The incident drew comparisons to a 2010 mining accident in Chile, which saw 33 miners trapped underground for 69 days before their rescue.
"It is a miracle," said Zeng Kunyuan on microblog Weibo, China's equivalent of Twitter.
"I hope they can sit at the dinner table on the eve of Spring Festival!" the posting said, referring to the upcoming Chinese New Year when people return home to have a traditional meal with their families.
Dramatic footage released by CCTV showed rescue crews applauding as the men were brought above ground in China's eastern province of Shandong.
Of the 29 that were trapped, one has been found dead and 11 were rescued the day after the accident. There has been no contact with the remaining 13, however.
"We used life detection equipment to search for them. Unfortunately, we haven't heard from those 13 missing miners," Zhang Shuping, mayor of Linyi city which governs the area was quoted by CCTV as saying.
The four rescued men were shown being wrapped in military blankets, blindfolded to protect their eyes, and put into ambulances. They were named by state media as Zhao Zhicheng, aged 50, Li Qiusheng, 39, Guan Qingji, 58, and Hua Mingxi, 36.
"Thanks," Guan was reported to have said as he was hoisted out clinging to a cable which held him with a series of harnesses strapped to his body.
The four had suffered no major injuries and would soon be able to return home, the official Xinhua news agency quoted Cao Qingde, deputy head of the local hospital where they were being treated, as saying.
Rescuers first detected signs of life on December 30. The rescue team managed to contact the miners, the broadcaster reported, and send down food, clothes and lamps through a tunnel.
However, complicated geological conditions made the rescue difficult with crews having to account for the structural instability of the tunnel and falling rocks.
Government officials said this marked China's first rescue of its kind, accomplished by drilling through rock with a large drill head.
"In the rescue history of the People's Republic of China, this has turned a new page," Gao Guangwei of the State Administration of Work Safety told CCTV.
The incident was the latest deadly accident in a country where safety rules are often flouted to cut costs.
Despite the jubilation over the rescue, some online postings called for accountability over the accident.
"Feel so proud of this? Shouldn't it be the time to hold someone accountable?" said one posting.
The mine owner committed suicide by drowning himself at the scene soon after the collapse, Xinhua reported previously.
The cause of the collapse is under investigation, but industrial safety regulations are often evaded in China and corruption enables bosses to pursue profits at the cost of worker safety.
Four officials in Pingyi county, where the mine is located, including the county's party chief and head of government, were removed from their posts in the wake of the incident.
The gypsum pit and other mines in its vicinity were ordered to stop production in October by local authorities because of a risk of sinkholes, but it kept operating secretly, the Beijing Times reported earlier.
Accidents linked to lax industrial safety enforcement saw hundreds of people killed in China last year, including a landslide caused by improper disposal of waste in the southern commercial hub of Shenzhen in December and chemical blasts in the industrial city of Tianjin in August.