Severe haze episodes have hit Beijing many times in the past few years, especially the “crazy bad” pollution described by the US Embassy in Beijing. The publication of numerous multimedia reports on the severe haze has increased awareness among the people in China regarding air pollution and PM2.5. It is assumed that the severe haze occurred suddenly for unclear reasons. In this context, long-term evaluation of the air pollution in Beijing is necessary. Through hourly and daily PM2.5 concentration records, meteorological datasets from August 2004 onward, and the Kolmogorov–Zurbenko (KZ) filter approach, the evolutions of the long-term components (or background values) of PM2.5 concentrations at an urban and a rural station in Beijing were statistically analyzed, and the possible causes of variation in the trends of these components were evaluated in this study. The long-term components of PM2.5 concentrations decreased significantly at both the urban (–3.40 µg m–3 y–1) and rural (–1.16 µg m–3 y–1) stations. The most serious pollution predominantly occurred in an earlier period than recent years, when little attention was being paid. The decrease in PM2.5 concentration was mainly attributed to the reduction in pollutant emission, despite the distinct increase in total energy consumption and motor vehicle use. However, the unfavorable climate changes (i.e., a reduction in wind speed and an increase in relative humidity) reduced the efficiency of atmospheric environmental governance. Because of the unfavorable climate or meteorological changes, the trends of PM2.5 reduction derived from the pollutant emission controls have been offset by up to approximately 15% in both urban and rural areas of Beijing during the last decade.