Severe haze episodes hit Beijing many times in the past few years, especially the “crazy bad” pollution described by the Beijing US Embassy. The publication of numerous multimedia reports on the severe haze lead to increase awareness among the people in China regarding air pollution and PM2.5. It is assumed that the severe haze occurred suddenly and it was not clear why. In this context, long-term evaluation of the air pollution in Beijing is necessary. Through hourly and daily PM2.5 concentration records, meteorological datasets from August 2004 onward, and the Kolmogorov–Zurbenko (KZ) filter approach, the evolutions of the long-term components (or background values) of PM2.5 concentrations at an urban and a rural station of Beijing were statistically analyzed and the possible causes for the trend variation of the long-term components were evaluated in this study. The long-term components of PM2.5 concentrations decreased significantly at both the urban (–3.40 µg m–3 y–1) and rural (–1.16 µg m–3 y–1) stations. Compared with the recent years, the most serious pollution predominantly occurred in the early period when too little attention was being paid. The decrease in PM2.5 concentration was mainly attributed to the reduction in pollutant emission, despite the distinct increase in total energy consumption and motor vehicle use. However, the unfavorable climate changes (i.e., reduction in wind speed and increase in relative humidity) depressed the efficiency of the atmospheric environmental governance. Because of the unfavorable climate or meteorological condition changes, the trends of PM2.5 reduction derived from the pollutant emission controls were offset by up to approximately 15% in both urban and rural areas of Beijing during the last decade.