China's carbon dioxide (CO2) emission ranks highest in the world. China is committed to reduce its CO2 emission by 40% to 45% from the 2005 levels by 2020. To fulfill the target, China's CO2 emission reduction must exceed 6995 million tons. Energy consumption and CO2 emission of China's urban district heating (UDH) are increasing. The current policy implemented to improve UDH focuses on replacing coal with natural gas to reduce energy consumption and CO2 emission to some extent. This paper proposes that heat pump heating (HPH) could serve as a replacement for UDH to help realize energy‐saving and emission‐reduction goals to a greater extent. The paper also analyzes the impact of this replacement on the heating and power generation sectors. The results show that replacing coal‐based UDH with HPH decreases energy consumption and CO2 emission by 43% in the heating sector. In the power generation sector, the efficiency of power generation at the valley electricity time increases by 0.512%, and the ratio of peak–valley difference decreases by 16.5%. The decreases in CO2 emission from the heating and power generation sectors cumulatively account for 5.55% of China's total CO2 emission reduction target in 2020.