Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5
Particulate matter 2.5 (PM₂.₅) pollution is an actual problem in the modern world and forecasting of the daily concentration of PM₂.₅ is a challenging task for researchers. In this study, a novel neural network model that effectively forecasts daily PM₂.₅ in Hangzhou city was developed in the form...
Збережено в:
| Опубліковано в: : | Реєстрація, зберігання і обробка даних |
|---|---|
| Дата: | 2017 |
| Автори: | , , , |
| Формат: | Стаття |
| Мова: | Англійська |
| Опубліковано: |
Інститут проблем реєстрації інформації НАН України
2017
|
| Теми: | |
| Онлайн доступ: | https://nasplib.isofts.kiev.ua/handle/123456789/131685 |
| Теги: |
Додати тег
Немає тегів, Будьте першим, хто поставить тег для цього запису!
|
| Назва журналу: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Цитувати: | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 / Minglei Fu, Chen Wang, Zichun Le, D. Manko // Реєстрація, зберігання і обробка даних. — 2017. — Т. 19, № 3. — С. 53-64. — Бібліогр.: 29 назв. — англ. |
Репозитарії
Digital Library of Periodicals of National Academy of Sciences of Ukraine| _version_ | 1860112096239288320 |
|---|---|
| author | Minglei Fu Chen Wang Zichun Le Manko, D. |
| author_facet | Minglei Fu Chen Wang Zichun Le Manko, D. |
| citation_txt | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 / Minglei Fu, Chen Wang, Zichun Le, D. Manko // Реєстрація, зберігання і обробка даних. — 2017. — Т. 19, № 3. — С. 53-64. — Бібліогр.: 29 назв. — англ. |
| collection | DSpace DC |
| container_title | Реєстрація, зберігання і обробка даних |
| description | Particulate matter 2.5 (PM₂.₅) pollution is an actual problem in the modern world and forecasting of the daily concentration of PM₂.₅ is a challenging task for researchers. In this study, a novel neural network model that effectively forecasts daily PM₂.₅ in Hangzhou city was developed in the form of a restricted Boltzmann machines double layer back propagation neural network model (RBM-DL-BPNN). Air quality index, the air pollutants, e.g., particulate matter 10 (PM10), PM₂.₅, SO₂, CO, NO₂, O₃, and meteorological parameters (temperature, dew point, humidity, pressure, wind speed, and precipitation) of Hangzhou city were used in this study to train and test three models: RBM-DL-BPNN, double layer back propagation neural network (DL-BPNN), and back propagation neural network (BPNN). The results of experiments and analyses performed indicate that RBM-DL-BPNN has a smaller mean absolute percent error (MAPE), smaller overall daily absolute percentage errors, and more results in terms of absolute percentage error within the range 0-50 % than DL-BPNN and BPNN.
Загрязнение ультрадисперсными частицами (УДЧ) класса PM₂.₅ является актуальной проб-лемой в современном мире. Прогнозирование их ежедневной концентрации является сложной задачей для исследователей. Разработана новая модель в виде ограниченной машины Больцмана обратной связи с удвоенным слоем (RBM-DL-BPNN). Эффективность предложенной модели показана на примере прогнозирования концентрации PM₂.₅ в городе Ханчжоу. Показатели качества воздуха, его загрязнения (PM10, УДЧ PM₂.₅, SO₂, CO, NO₂, O₃), метеорологические параметры (сред-несуточная температура, точка росы, влажность, атмосферное давление, скорость ветра и количество осадков) в Ханчжоу использованы в работе для обучения и тестирования трех моделей: RBM-DL-BPNN, нейронной сети с обратной связью с двойным слоем (DL-BPNN) и нейронной сети обратного распространения (BPNN). Результаты проведенных исследований показали, что относительная погрешность результатов использования RBM-DL-BPNN является наименьшей среди изученных нейронных сетей, которая заключается в том, что количество значений этой погрешности в диапазоне 0–50 % для RBM-DL-BPNN значительно больше, чем в случаях DL-BPNN и BPNN.
Забруднення ультрадисперсними частинками (УДЧ) класу PM₂.₅ є актуальною проблемою у сучасному світі. Прогнозування їхньої щоденної концентрації є складним завданням для дослідників. Розроблено нову модель у вигляді обмеженої машини Больцмана зворотного зв’язку з подвоєним шаром (RBM-DL-BPNN). Ефективність запропонованої моделі показано на прикладі прогнозування концентрації УДЧ РМ₂,₅ у місті Ханчжоу. Показники якості повітря, його забруднення (РМ10, РМ₂,₅, SO₂, CO, NO₂, O₃), метеорологічні параметри (середньодобова температура, точка ро-си, вологість, атмосферний тиск, швидкість вітру та кількість опадів) у Ханчжоу використано в роботі для навчання та тестування трьох моделей: RBM-DL-BPNN, нейронної мережі зі зворотним зв’язком з подвійним шаром (DL-BPNN) і нейронної мережі зворотного поширення (BPNN). Результати проведених досліджень показали, що відносна похибка результатів використання RBM-DL-BPNN є найменшою серед вивчених нейронних мереж, яке полягає в тому, що кількість значень цієї похибки в діапазоні 0–50 % для RBM-DL-BPNN значно більше, ніж для випадків DL-BPNN і BPNN.
|
| first_indexed | 2025-12-07T17:34:27Z |
| format | Article |
| fulltext |
ISSN 1560-9189 , , 2017, . 19, 3 53
004.032.26
Minglei Fu1, Chen Wang1, Zichun Le1, Dmytro Manko2
1College of Sciences, Zhejiang University of Technology
288 Liuhe Road, 310023 Hangzhou, China
2Institute for Information Recording, National Academy of Sciences of Ukraine
. Shpak, 2 st., 03113 Kyiv, Ukraine
Double layer back propagation neural network
based on restricted Boltzmann machines
for forecasting daily particulate matter 2.5
Particulate matter 2.5 (PM2.5) pollution is an actual problem in the modern
world and forecasting of the daily concentration of PM2.5 is a challenging
task for researchers. In this study, a novel neural network model that effec-
tively forecasts daily PM2.5 in Hangzhou city was developed in the form of a
restricted Boltzmann machines double layer back propagation neural net-
work model (RBM-DL-BPNN). Air quality index, the air pollutants, e.g.,
particulate matter 10 (PM10), PM2.5, SO2, CO, NO2, O3, and meteorological
parameters (temperature, dew point, humidity, pressure, wind speed, and
precipitation) of Hangzhou city were used in this study to train and test three
models: RBM-DL-BPNN, double layer back propagation neural network
(DL-BPNN), and back propagation neural network (BPNN). The results of
experiments and analyses performed indicate that RBM-DL-BPNN has a
smaller mean absolute percent error (MAPE), smaller overall daily absolute
percentage errors, and more results in terms of absolute percentage error
within the range 0–50 % than DL-BPNN and BPNN.
Key words: Neural network, restricted Boltzmann machines, Particulate
matter 2.5, Forecasting.
1. Introduction
With sustainable growth of a social economy and rapid expansion of urban popu-
lations, urban air pollution problems have become increasingly serious [1, 2]. Among
various kinds of air pollutants, particulate matter 2.5 (PM2.5) is the main pollution in
Hangzhou city [3]. Nowadays, merely measuring urban PM2.5 is not enough. Under-
standing the development trend of PM2.5 concentration to prevent air pollution in cities
and guarantee the health of urban residents is a task of vital importance.
In recent years, artificial neural networks (ANNs) have been proven effective in
forecasting trends in air pollution such as predicting CO ambient concentration [4] and
© Minglei Fu, Chen Wang, Zichun Le, Dmytro Manko
Minglei Fu, Chen Wang, Zichun Le, Dmytro Manko
54
predicting hourly PM2.5 concentration [5]. Furthermore, various models have success-
fully optimized ANNs [6–20]. In general, the following methods can be used to optimize
ANNs: input data selection, algorithm optimization, and model combination. Some re-
searchers have already conducted studies and analyzed the best choices for input data to
optimize the effectiveness of ANNs. For example, Voukantsis et al. used principal
component analysis (PCA) to reduce the dimension of the original input data and trans-
form the original input dataset to a linear combination in order to optimize the artificial
neural networks multilayer perceptron (ANN-MLP) model [6]. Gennaro et al. performed
sensitivity analysis to understand the importance of different variables in developing an
ANN [8]. Antanasijevi et al. selected and optimized the input data to an ANN using a
genetic algorithm [10]. In addition, some researchers have incorporated techniques such
as fuzzy logic, k-means clustering, chaotic particle swarm optimization (CPSO), and
wavelet transformation into neural networks to optimize ANNs. For instance, Mishra et
al. combined a neural network and fuzzy logic to forecast PM2.5 during haze conditions
[11]. Elangasinghe et al. combined ANN with k-means clustering to analyze PM10 and
PM2.5 [13]. He et al. created a novel hybrid model combining ANN and CPSO to im-
prove forecasting accuracy [14]. Siwek and Osowski combined wavelet transformation
and neural network to forecast the daily average concentration of PM10 [15]. Researchers
have also combined ANNs with other models to create new models that can be used for
more accurate forecasting. For example, Perez et al. combined a nearest neighbor model
(NNM) with ANN to improve the accuracy of PM10 concentration forecasting [16].
D z-Robles et al. created a novel hybrid model combining Box-Jenkins Time Series
(ARIMA) and ANN and improved the forecast accuracy of particulate matter [18].
Al-Alawi et al. combined principal component regression (PCR) and ANN to predict
ozone concentration levels in the lower atmosphere [20].
The methods cited above have been proved to effectively improve the performance
of ANNs. However, the ANN models proposed in previous works were usually ANNs
with only a single hidden layer. Hence, the prediction accuracy of the models might be
restricted by the inherent shortcomings of the single layer ANNs. For example, it is
usually difficult to optimize the weights of the neurons in single layer ANNs to obtain
higher prediction accuracy. This study focused on optimization of a double layer back
propagation neural network (DL-BPNN) for forecasting daily PM2.5 using restricted
Boltzmann machines (RBM). This is in contrast to the previous works cited above,
which focused primarily on the input parameters and models to make the prediction of
ANNs more accurate. RBM, which can learn input data features, is used to train the
weights that initialize the DL-BPNN. The proposed RBM-DL-BPNN model was evalu-
ated by comparing its results in predicting the PM2.5 concentration in Hangzhou city to
those obtained from the standard DL-BPNN and BPNN models. Further, its advantages
and disadvantages were analyzed.
2. Material and methods
2.1. Data collection
In recent years, concern about the air quality of Chinese cities has been increasing.
Hangzhou city covers an area of 16596 square kilometers and had an estimated popula-
tion of approximately 9 million people in 2015 [21]. In 2014, there were 137 pollution
Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5
ISSN 1560-9189 , , 2017, . 19, 3 55
days in Hangzhou and 93 days with PM2.5 as the primary pollutant [3]. Therefore, the air
quality daily data and the meteorological daily data of Hangzhou from December 2013 to
August 2016 were both used in this study. The data used include the air quality index
(AQI), concentration of PM2.5, PM10, SO2, CO, NO2, and O3 in the air, temperature, dew
point, humidity, pressure, wind speed, and precipitation. The data from December 2013
to May 2016 were used for training, whereas the data from June 2016 to August 2016
were used to test the models. The air quality daily data were collected from PM2.5
monitoring network websites [22] and the meteorological daily data were collected from
the Weather Underground websites [23]. Before starting, the input data were normalized.
Subsequently, the value of the input data was transformed into (0,1–0,9), which is helpful
for training. The following equation was used:
min
max min
0,8* 0,1
x x
x
x x
, (1)
where x is the original data and xmin and xmax are the minimum and maximum values,
respectively. x* is the normalized data.
2.2. Back propagation neural network (BPNN)
ANNs are mathematical structures consisting of a number of interconnected neu-
rons. An ANN is able to emulate the process that people use to recognize patterns, ac-
quire knowledge, and solve problems [1]. A BPNN is a classic neural network with three
layers: an input layer, a hidden layer, and an output layer, as shown in Fig. 1. The
working principle of a BPNN can be divided into two processes. In the first process,
called the signal forward propagation process, the training data are introduced from the
input layer, propagated through the hidden layer, and finally outputted from the output
layer. The neurons in the hidden layer sum the weighted arriving signal:
*
1 1
1
n
j ij i
i
h w x b , (2)
where h1j (j = 1,2,...m) is the output value of the hidden layer neuron, w1ij (i = 1,2,...,n) are
the weights between the input layer and the hidden layer, xi
* (i = 1,2,...,n) are the nor-
malized input data, and b is a bias value [24].
Fig. 1. Structure of a back propagation neural network (BPNN)
Minglei Fu, Chen Wang, Zichun Le, Dmytro Manko
56
The second process is signal back propagation. In this process, errors in the output
values and the actual values are propagated back to the hidden layer to adjust the weights
between the hidden layer and the output layer. Similarly, the weights between the input
layer and the hidden layer are adjusted when the error returns. BPNN is considered as
being completely trained when the error has been reduced to a stable value. After training
the BPNN, test data can be introduced to forecast daily PM2.5 concentration.
2.3. Double layer BPNN (DL-BPNN)
BPNNs can be used to solve nonlinear problems. The structure of a double layer
back propagation neural network is similar to that of a BPNN. However, DL-BPNN has
two hidden layers, as shown in Fig. 2. The equations utilized by the first and the second
hidden layers are:
1 1
1
n
j ij i
i
h w x b , (3)
2 2 1
1
m
l jl j
j
h w h b , (4)
where h'1j (j = 1,2,...m) is the output value of the first hidden layer; h'2l (l = 1,2,...k) is the
output value of the second hidden layer; w'1ij (i = 1,2,...,n) are the weights between the
input layer and the first hidden layer; w'2jl (j = 1,2,...,m) are the weights between the first
hidden layer and the second hidden layer; xi
* (i = 1,2,...,n) are the normalized input data,
and b is the bias value.
Fig. 2. Structure of a double layer back propagation neural network (DL-BPNN)
Theoretically, DL-BPNN is more useful than BPNN for solving nonlinear prob-
lems. However, DL-BPNN may in fact not be able to show its advantages because of the
error attenuation during the error back propagation process. Consequently, weights ad-
justment between the input layer and the first hidden layer is small. However, the signal
forward propagation process begins with the input layer and the first hidden layer, which
means that regardless of how the second hidden layer is trained, the output values are
confused by the first two layers and the entire network will operate poorly. As a result,
DL-BPNNs are not widely used as BPNNs.
Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5
ISSN 1560-9189 , , 2017, . 19, 3 57
2.4. Double layer back propagation neural network
based on restricted Boltzmann machines (RBM-DL-BPNN)
Deep neural networks (DNNs) were introduced in 2006 and have been successfully
applied in speech recognition and image recognition [25, 26, 27]. RBM is an important
component of any DNN, as it can learn the features of input data through unsupervised
training [27]. Random weight initialization of a DL-BPNN fails with very high prob-
ability in the basin of attraction of a poor local minimum [28]. Thus, an RBM is used to
pre-train the weights of DL-BPNN [29]. The training of the RBM-DL-BPNN can be
divided into two parts. In the first part, RBM is used to learn the features of the input data.
An RBM has a visible layer and several hidden layers but no visible-visible or hid-
den-hidden connections. In a binary RBM, the weights on connections and the biases of
individual units define a probability distribution over the joint states of the visible and
hidden units via an energy function. The energy of a joint configuration is given as fol-
lows [26]:
=1 =1 =1 =1
, =
V H V H
ij i j i i j j
i j i j
E v h w v h b v a h , (5)
where = (w, b, a) and wij represents the symmetric interaction term between visible unit
i and hidden unit j while bi and aj are their bias terms. V and H are the numbers of visible
and hidden units.
The aim of RBM training is to learn the parameter = (w, b, a), whereas the value
of parameter w is desired. The structure of RBM-DL-BPNN is shown in Fig. 3. In the
figure, the unsupervised structure RBM 1 trains the weights between the input layer and
the first hidden layer (W1 ) of RBM-DL-BPNN. Then, W1 is used as the initial weight to
train the weights between the first hidden layer ad the second hidden layer (W2 ) of
RBM-DL-BPNN through a double layer unsupervised structure, RBM 2.
Fig. 3. Structure of the double layer back propagation neural network model
based on restricted Boltzmann machines (RBM-DL-BPNN)
Minglei Fu, Chen Wang, Zichun Le, Dmytro Manko
58
Then, the weights between the output layer and the second hidden layer (W3 ) are
trained and W1 and W2 are fine-tuned. The W1 and W2 trained by RBM 1 and RBM 2
are used as the initial weights of the first two layer weights of the RBM-DL-BPNN
model. Then, we use supervised training to finally train W3 and adjust W1 and W2 .
3. Results and discussion
3.1. Results
The efficiency of all three models presented above was evaluated using efficiency
indexes: root mean square error (RMSE) (6), mean absolute error (MAE) (7), and mean
absolute percent error (MAPE) (8). RMSE and MAE were used to evaluate the reliability,
whereas MAPE was used to evaluate the accuracy of the models [1, 18]:
2
1
1 n
i i
i
RMSE t y
n
, (6)
1
1 ( )
n
i i
i
MAE t y
n
, (7)
1
| |1 ( ) 100 %
n
i i
i i
t yMAPE
n t
, (8)
where n is the number of data points, yi is the predicted value, and ti is the actual ob-
served value.
There are 33 months from December 2013 to August 2016. The data for the first 30
months were used to train models and the data for the last three months were used to test
them. Thus, 912 pieces of data were used for training and 90 pieces of data for testing.
Figs. 4–6 show curves of the daily PM2.5 concentration forecasted by three models and
the actual data in June, July, and August 2016. The black, red, blue, and green curves
represent actual data, RBM-DL-BPNN forecasted data, DL-BPNN forecasted data, and
BPNN forecasted data, respectively. On the whole, the trends of the three models are all
similar to the actual data. However, the trend of the red curve is closer to the trend of the
black curve than the trends of the blue and green curves. Further, the blue curve is closer
to the black curve than to the green curve, which means that, to some extent, the
DL-BPNN model can forecast more accurately than the BPNN model. Further, the
RBM-DL-BPNN model can forecast more accurately than both the DL-BPNN model
and the BPNN model.
Table 1 shows the efficiency indexes of BPNN, DL-BPNN, and RBM-DL-BPNN
in forecasting PM2.5 from June 2 to August 31, 2016 in Hangzhou city. From Table 1 it is
clear that the MAPE of RBM-DL-BPNN is lower than that of both DL-BPNN and
BPNN. Further, the MAPE of DL-BPNN is lower than that of BPNN. RBM-DL-BPNN’s
RMSE and MAE are sometimes higher than those of the other two models; however, the
difference is minimal. From Figs. 4–6 and Table 1, it is clear that the DL-BPNN model
actually has several advantages over the BPNN model and that RBM is useful for the
DL-BPNN model.
Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5
ISSN 1560-9189 , , 2017, . 19, 3 59
Table 1. Efficiency indexes of BPNN, DL-BPNN, and RBM-DL-BPNN for PM2.5
from June 2016 to August 2016 in Hangzhou city
BPNN DL-BPNN RBM-DL-BPNN
RMSE MAE MAPE RMSE MAE MAPE RMSE MAE MAPE
June 12,941 10,348 36,353 % 11,498 9,179 31,941 % 12,710 9,335 28,054 %
July 9,666 7,413 31,376 % 9,753 7,613 31,295 % 10,612 8,253 26,468 %
August 9,620 7,949 40,148 % 10,981 8,930 38,912 % 7,835 6,602 29,484 %
0 5 10 15 20 25 30
10
20
30
40
50
60
70
80
D
ai
ly
P
M
2,
5 c
on
ce
nt
ra
tio
n
(
g/
m
3 )
Date (day)
Actual data
BPNN forecasted data
DL-BPNN forecasted data
RBM-DL-BPNN forecasted data
Fig. 4. Actual data for June 2016 versus the daily PM2.5 concentration forecasted by the three models
0 5 10 15 20 25 30
10
20
30
40
50
60
Actual data
BPNN forecasted data
DL-BPNN forecasted data
RBM-DL-BPNN forecasted data
Date (day)
D
ai
ly
P
M
2,
5 c
on
ce
nt
ra
tio
n
(
g/
m
3 )
Fig. 5. Actual data for July 2016 versus the daily PM2.5 concentration forecasted by the three models
Minglei Fu, Chen Wang, Zichun Le, Dmytro Manko
60
0 5 10 15 20 25 30 35
0
10
20
30
40
50
Date (day)
D
ai
ly
P
M
2,
5 c
on
ce
nt
ra
tio
n
(
g/
m
3 )
Actual data
BPNN forecasted data
DL-BPNN forecasted data
RBM-DL-BPNN forecasted data
Fig. 6. Actual data for August 2016 versus the daily PM2.5 concentration forecasted by the three models
3.2. Discussion
In order to deeply analyze the improvements attributable to the RBM-DL-BPNN
model, we collected the daily absolute percentage error of the three models and calcu-
lated the days from different error ranges.
Figs. 7 shows the distribution of MAPE of the three models, in which the dense,
sparse, and none filling of a column represents a distribution of the daily MAPE of the
RBM-DL-BPNN model, DL-BPNN model, and BPNN model, respectively — compares
the absolute percentage error in June-August 2016 of the three models. The distribution
evidence, that the BPNN model is characterized by bigger number of relative errors less
than 10 %. At the same time RBM-DL-BPNN model are characterized by slightly lower
amount of relative errors less than 10 %, but the amount of relative errors less than 30 %
higher than in the case of another two models.
0 50 100 150 200
0
10
20
30
40
BPNN forecasted data
DL-BPNN forecasted data
RBM-DL-BPNN forecasted data
C
ou
nt
s
MAPE, %
Fig. 7. Absolute percentage error of the three models in June 2016
Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5
ISSN 1560-9189 , , 2017, . 19, 3 61
The biggest relative errors are 88,6 % (dense filled column), 120,9 % (sparse filled
column), and 154,8 % (none filled column), respectively. Considering the results pre-
sented in Figs. 7, the largest errors of the RBM-DL-BPNN model are all lower than those
of the DL-BPNN model and the BPNN model. Furthermore, intuitively, the overall daily
absolute percentage errors of the RBM-DL-BPNN model are lower than those of the
DL-BPNN model and the BPNN model, which makes the RBM-DL-BPNN model more
credible than the others.
Table 2. shows the days of the three models’ results in different error ranges. From
Table 2, it is clear that the results of the three models are primarily distributed within the
range 0–50 %. In the 0–50 % range, the RBM-DL-BPNN model has more data points
than the other two models, whereas they are all close in the 50–100 % range. Con-
spicuously, in the >100 % range, the RBM-DL-BPNN model has zero data points,
whereas the DL-BPNN model and the BPNN model both have seven data points. Thus,
from the above analysis, we can conclude that the RBM-DL-BPNN model is able to
forecast more accurately than the other models.
Table 2. Number of days spent in the three different absolute
percentage error scales by each of the three models
BPNN DL-BPNN RBM-DL-BPNN
0–50 % 50–100 % >100 % 0–50 % 50–100 % >100 % 0–50 % 50–100% >100 %
June 23 3 3 23 4 2 25 4 0
July 25 4 2 25 3 3 28 3 0
August 22 7 2 24 5 2 26 5 0
Sum 70 14 7 72 12 7 79 12 0
However, the forecasting accuracy of the RBM-DL-BPNN model is not suffi-
ciently satisfying. Two main factors could account for this result. Firstly, the data used in
this work are data taken from websites; they represent the mean values of the meteoro-
logical parameters of the entire Hangzhou city, which makes the reliability of the rela-
tionships between variables weak. Secondly, we could not collect all the necessary data
concerning PM2.5 concentration in this
work. The pollution sources of PM2.5
and their proportions in Hangzhou city
are shown in Fig. 10 [3].
Traffic pollution, industrial pol-
lution, and dust and coal pollution are
the top four pollution sources of PM2.5
in Hangzhou city. However, obtaining
these data directly from official statis-
tics is difficult. However, we will try to
collect the related data in future work.
Fig. 8. Pollution sources of PM2.5 in Hangzhou [3]
Minglei Fu, Chen Wang, Zichun Le, Dmytro Manko
62
4. Conclusions
ANNs are widely used to process air quality and meteorological records, and many
optimized neural networks have proved effective. In this study, we proposed and devel-
oped the RBM-DL-BPNN model for forecasting daily PM2.5 concentration. The model
uses RBM to learn features of the input data and saves the information in weights to ini-
tialize the weights of the DL-BPNN model. RBM is firstly applied to optimize the pre-
diction of the ANN, which makes the DL-BPNN more effective. The meteorological
parameters of Hangzhou city for the period December 2013 to May 2016 were used to
train three models: RBM-DL-BPNN, DL-BPNN and BPNN and the remainder from
June 2016 to August 2016 used for testing. Experimental results and analysis show that
the RBM-DL-BPNN model has a smaller MAPE, smaller daily absolute percentage er-
rors on the whole, and no errors above 100 %. Thus, it can be concluded that the
RBM-DL-BPNN model can relatively accurately and reliably forecast daily PM2.5 con-
centration for Hangzhou city. Although the RBM-DL-BPNN model is better than the
DL-BPNN and BPNN models, the RBM-DL-BPNN model sometimes could not forecast
accurately because of uncertain anthropogenic factors and cases of extreme weather
conditions. In the future, we will study the relationship between daily PM2.5 concentra-
tion and anthropogenic factors so that human activity and extreme weather conditions
can be added as parameters in an appropriate manner for more accurate forecasting.
Acknowledgments
This work was financially supported by the Special Funding of «the Belt and
Road» International Cooperation of Zhejiang Province (2015C04005) and National
Natural Science Foundation of China (61571399).
1. Minglei Fu., Weiwen Wang, Zichun Le, Mahdi Safaei Khorram. Prediction of particular matter
concentration by developed feed-forward neural network with rolling mechanism and gray model. Neural.
Comput. & Applic. 2015. Vol. 26. Iss. 8. P. 1789–1797.
2. Yifeng Xue, Hezhong Tian, Jing Yan [et al.]. Present and future emissions of HAPs from cre-
matories in China. Atmos. Environ. 2016. Vol. 124. P. 28–36
3. Hangzhou released PM2.5 source analysis results: motor vehicle exhaust emissions first. URL:
http://zjnews.zjol.com.cn/system/2015/06/06/020686281.shtml
4. Perez-Roa R., Castro J., Jorquera H., Perez-Correa J.R., Vesovic V. Air-pollution modelling in
an urban area: Correlating turbulent diffusion coefficients by means of an artificial. Atmos. Environ. 2006.
Vol. 400. Iss. 1. P. 109–125.
5. Perez P., Gramsch E. Forecasting hourly PM2.5 in Santiago de Chile with emphasis on night
episodes. Atmos. Environ. 2016. Vol. 124. P. 22–27.
6. Voukantsis D., Karatzas K., Kukkonen J., Räsänen T., Karppinen A., Kolehmainen K. Inter-
comparison of air quality data using principal component analysis, and forecasting of PM10 and PM2.5
concentrations using artificial neural networks, in Thessaloniki and Helsinki. Sci. Total Environ. 2011. Vol.
409. P. 1266–1276.
7. Ul-Saufie A.Z., Yahaya A.S., Ramli N.A., Rosaida N., Hamid H.A. Future daily PM10 concen-
trations prediction by combining regression models and feedforward backpropagation models with prin-
ciple component analysis (PCA). Atmos. Environ. 2013. Vol. 77. P. 621–630.
http://zjnews.zjol.com.cn/system/2015/06/06/020686281.shtml
Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5
ISSN 1560-9189 , , 2017, . 19, 3 63
8. Gennaro G.D., Trizio L., Gilio A.D. Neural network model for the prediction of PM10 daily
concentrations in two sites in the Western Mediterranean. Sci. Total Environ. 2013. Vol. 463–464.
P. 875–883.
9. Shan S.Q., Feng L., Jian Z.W., Beibei S. Analysis and forecasting of the particulate matter (PM)
concentration levels over four major cities of China using hybrid models. Atmos. Environ. 2014. Vol. 98.
P. 665–675.
10. Antanasijevi D.Z., Pocajt V.V., Povrenovi D.S., Risti M.D., Peri -Gruji A.A. PM10 emission
forecasting using artificial neural networks and genetic algorithm input variable optimization. Sci. Total
Environ. 2014. Vol. 443. P. 511–519.
11. Mishra D., Goyal P., Upadhyay A. Artificial intelligence based approach to forecast PM2.5
during haze episodes: A case study of Delhi, India. Atmos. Environ. 2015. Vol. 102. P.239–248.
12. Yildirim Y., Bayramoglu M. Adaptive neuro-fuzzy based modelling for prediction of air pollu-
tion daily levels in city of Zonguldak. Chemosphere. 2006. Vol. 63. P. 1575–1582.
13. Elangasinghe M.A., Singhal N., Dirks K.N., Salmond J.A., Samarasinghe S. Complex time se-
ries analysis of PM10 and PM2.5 for a coastal site using artificial neural network modelling and k-means
clustering. Atmos. Environ. 2014. Vol. 94. P. 106–116.
14. He H.D., Lu W.Z., Yu X. Prediction of particulate matter at street level using artificial neural
networks coupling with chaotic particle swarm optimization algorithm. Build. Environ. 2014. Vol. 78.
P. 111–117.
15. Siwek K., Osowski S. Improving the accuracy of prediction of PM10 pollution by the wavelet
transformation and an ensemble of neural predictors. Eng Appl Artif Intell. 2012. Vol. 25. P. 1246–1258.
16. Perez P. Combined model for PM10 forecasting in a large city. Atmos. Environ. 2012. Vol. 60.
P. 271–276.
17. Arhami M., Kamali N., Rajabi M.M. Predicting hourly air pollutant levels using artificial neu-
ral networks coupled with uncertainty analysis by Monte Carlo simulations. Environ Sci.Pollut. Res. 2013.
Vol. 20. P. 4777–4789.
18. D az-Robles L.A., Ortega J.C., Fu J.S. [et al.] A hybrid ARIMA and artificial neural networks
model to forecast particulate matter in urban areas: The case of Temuco, Chile. Atmos. Environ. 2008.
Vol.42. P. 8331–8340.
19. Zhou Q.P., Jiang H.Y., Wang J.Z., Zhou J.L. A hybridmodel for PM2.5 forecasting based on
ensemble empirical mode decomposition and a general regression neural network. Sci. Total Environ.
2014. Vol. 496. P. 264–274.
20. Al-Alawi S.M., Abdul-Wahab S.A., Bakheit C.S. Combining principal component regression
and artificial neural networks for more accurate predictions of ground-level ozone. Environ. Model. Softw.
2008. Vol. 23. P. 396–403.
21. The main data bulletin of 1 % population sample survey in Zhejiang Province. URL:
http://www.zj.stats.gov.cn/tjgb/rkcydcgb/201601/t20160128_168706.html
22. Air Quality Index Monthly Statistical Data. URL: http://www.aqistudy.cn/historydata/ month-
data.php?city=
23. , , , 2013. URL: https://www.wunderground.com/
history/airport/ZSHC/2013/12/2/DailyHistory.html
24. Liu D.J., Li L. Application Study of Comprehensive Forecasting Model Based on Entropy
Weighting Method on Trend of PM2.5 Concentration in Guangzhou, China. Int. J. Environ. Res. Public
Health. 2015. Vol. 12. P. 7085–7099.
25. Hinton G.E., Osindero S., Teh Y.W. A fast learning algorithm for deep belief nets. Neural
Comput. 2006. Vol. 18. P. 527–1554.
http://www.zj.stats.gov.cn/tjgb/rkcydcgb/201601/t20160128_168706.html
http://www.aqistudy.cn/historydata/
https://www.wunderground.com/
Minglei Fu, Chen Wang, Zichun Le, Dmytro Manko
64
26. Mohamed A., Dahl G.E., Hinton G. Acoustic Modeling using Deep Belief Networks. IEEE T
Audio Speech. 2012. Vol. 20. P. 14–22.
27. Hinton G., Li D., Yu D. [et al.] Deep Neural Networks for Acoustic Modeling in Speech Rec-
ognition. IEEE Signal Proc. Mag. 2012. Vol. 29. P. 82–97.
28. Dumitru Erhan,Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent
Erhan. Why Does Unsupervised Pre-training Help Deep Learning? Mach. Learn Res. 2010. Vol. 11.
P. 625.
29. Dahl G.E., Yu D., Member S., Li D., Acero A. Context-Dependent Pre-Trained Deep Neural
Networks for Large-Vocabulary Speech Recognition. IEEE T Audio Speech. 2012. Vol. 20. P. 30–42.
Recceived 26.07.2017
|
| id | nasplib_isofts_kiev_ua-123456789-131685 |
| institution | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| issn | 1560-9189 |
| language | English |
| last_indexed | 2025-12-07T17:34:27Z |
| publishDate | 2017 |
| publisher | Інститут проблем реєстрації інформації НАН України |
| record_format | dspace |
| spelling | Minglei Fu Chen Wang Zichun Le Manko, D. 2018-03-26T19:34:48Z 2018-03-26T19:34:48Z 2017 Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 / Minglei Fu, Chen Wang, Zichun Le, D. Manko // Реєстрація, зберігання і обробка даних. — 2017. — Т. 19, № 3. — С. 53-64. — Бібліогр.: 29 назв. — англ. 1560-9189 https://nasplib.isofts.kiev.ua/handle/123456789/131685 004.032.26 Particulate matter 2.5 (PM₂.₅) pollution is an actual problem in the modern world and forecasting of the daily concentration of PM₂.₅ is a challenging task for researchers. In this study, a novel neural network model that effectively forecasts daily PM₂.₅ in Hangzhou city was developed in the form of a restricted Boltzmann machines double layer back propagation neural network model (RBM-DL-BPNN). Air quality index, the air pollutants, e.g., particulate matter 10 (PM10), PM₂.₅, SO₂, CO, NO₂, O₃, and meteorological parameters (temperature, dew point, humidity, pressure, wind speed, and precipitation) of Hangzhou city were used in this study to train and test three models: RBM-DL-BPNN, double layer back propagation neural network (DL-BPNN), and back propagation neural network (BPNN). The results of experiments and analyses performed indicate that RBM-DL-BPNN has a smaller mean absolute percent error (MAPE), smaller overall daily absolute percentage errors, and more results in terms of absolute percentage error within the range 0-50 % than DL-BPNN and BPNN. Загрязнение ультрадисперсными частицами (УДЧ) класса PM₂.₅ является актуальной проб-лемой в современном мире. Прогнозирование их ежедневной концентрации является сложной задачей для исследователей. Разработана новая модель в виде ограниченной машины Больцмана обратной связи с удвоенным слоем (RBM-DL-BPNN). Эффективность предложенной модели показана на примере прогнозирования концентрации PM₂.₅ в городе Ханчжоу. Показатели качества воздуха, его загрязнения (PM10, УДЧ PM₂.₅, SO₂, CO, NO₂, O₃), метеорологические параметры (сред-несуточная температура, точка росы, влажность, атмосферное давление, скорость ветра и количество осадков) в Ханчжоу использованы в работе для обучения и тестирования трех моделей: RBM-DL-BPNN, нейронной сети с обратной связью с двойным слоем (DL-BPNN) и нейронной сети обратного распространения (BPNN). Результаты проведенных исследований показали, что относительная погрешность результатов использования RBM-DL-BPNN является наименьшей среди изученных нейронных сетей, которая заключается в том, что количество значений этой погрешности в диапазоне 0–50 % для RBM-DL-BPNN значительно больше, чем в случаях DL-BPNN и BPNN. Забруднення ультрадисперсними частинками (УДЧ) класу PM₂.₅ є актуальною проблемою у сучасному світі. Прогнозування їхньої щоденної концентрації є складним завданням для дослідників. Розроблено нову модель у вигляді обмеженої машини Больцмана зворотного зв’язку з подвоєним шаром (RBM-DL-BPNN). Ефективність запропонованої моделі показано на прикладі прогнозування концентрації УДЧ РМ₂,₅ у місті Ханчжоу. Показники якості повітря, його забруднення (РМ10, РМ₂,₅, SO₂, CO, NO₂, O₃), метеорологічні параметри (середньодобова температура, точка ро-си, вологість, атмосферний тиск, швидкість вітру та кількість опадів) у Ханчжоу використано в роботі для навчання та тестування трьох моделей: RBM-DL-BPNN, нейронної мережі зі зворотним зв’язком з подвійним шаром (DL-BPNN) і нейронної мережі зворотного поширення (BPNN). Результати проведених досліджень показали, що відносна похибка результатів використання RBM-DL-BPNN є найменшою серед вивчених нейронних мереж, яке полягає в тому, що кількість значень цієї похибки в діапазоні 0–50 % для RBM-DL-BPNN значно більше, ніж для випадків DL-BPNN і BPNN. This work was financially supported by the Special Funding of «the Belt and Road» International Cooperation of Zhejiang Province (2015C04005) and National Natural Science Foundation of China (61571399). en Інститут проблем реєстрації інформації НАН України Реєстрація, зберігання і обробка даних Технічні засоби отримання і обробки даних Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 Двошарова нейронна мережа зворотного поширення на основі обмежених машин Больцмана для прогнозування добової концентрації ультрадисперсних частинок РМ2.5. Двуслойная нейронная сеть обратного распространения на основе ограниченных машин Больцмана для прогнозирования суточной концентрации ультрадисперсных частиц РМ2.5 Article published earlier |
| spellingShingle | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 Minglei Fu Chen Wang Zichun Le Manko, D. Технічні засоби отримання і обробки даних |
| title | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 |
| title_alt | Двошарова нейронна мережа зворотного поширення на основі обмежених машин Больцмана для прогнозування добової концентрації ультрадисперсних частинок РМ2.5. Двуслойная нейронная сеть обратного распространения на основе ограниченных машин Больцмана для прогнозирования суточной концентрации ультрадисперсных частиц РМ2.5 |
| title_full | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 |
| title_fullStr | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 |
| title_full_unstemmed | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 |
| title_short | Double layer back propagation neural network based on restricted Boltzmann machines for forecasting daily particulate matter 2.5 |
| title_sort | double layer back propagation neural network based on restricted boltzmann machines for forecasting daily particulate matter 2.5 |
| topic | Технічні засоби отримання і обробки даних |
| topic_facet | Технічні засоби отримання і обробки даних |
| url | https://nasplib.isofts.kiev.ua/handle/123456789/131685 |
| work_keys_str_mv | AT mingleifu doublelayerbackpropagationneuralnetworkbasedonrestrictedboltzmannmachinesforforecastingdailyparticulatematter25 AT chenwang doublelayerbackpropagationneuralnetworkbasedonrestrictedboltzmannmachinesforforecastingdailyparticulatematter25 AT zichunle doublelayerbackpropagationneuralnetworkbasedonrestrictedboltzmannmachinesforforecastingdailyparticulatematter25 AT mankod doublelayerbackpropagationneuralnetworkbasedonrestrictedboltzmannmachinesforforecastingdailyparticulatematter25 AT mingleifu dvošarovaneironnamerežazvorotnogopoširennânaosnovíobmeženihmašinbolʹcmanadlâprognozuvannâdobovoíkoncentracííulʹtradispersnihčastinokrm25 AT chenwang dvošarovaneironnamerežazvorotnogopoširennânaosnovíobmeženihmašinbolʹcmanadlâprognozuvannâdobovoíkoncentracííulʹtradispersnihčastinokrm25 AT zichunle dvošarovaneironnamerežazvorotnogopoširennânaosnovíobmeženihmašinbolʹcmanadlâprognozuvannâdobovoíkoncentracííulʹtradispersnihčastinokrm25 AT mankod dvošarovaneironnamerežazvorotnogopoširennânaosnovíobmeženihmašinbolʹcmanadlâprognozuvannâdobovoíkoncentracííulʹtradispersnihčastinokrm25 AT mingleifu dvusloinaâneironnaâsetʹobratnogorasprostraneniânaosnoveograničennyhmašinbolʹcmanadlâprognozirovaniâsutočnoikoncentraciiulʹtradispersnyhčasticrm25 AT chenwang dvusloinaâneironnaâsetʹobratnogorasprostraneniânaosnoveograničennyhmašinbolʹcmanadlâprognozirovaniâsutočnoikoncentraciiulʹtradispersnyhčasticrm25 AT zichunle dvusloinaâneironnaâsetʹobratnogorasprostraneniânaosnoveograničennyhmašinbolʹcmanadlâprognozirovaniâsutočnoikoncentraciiulʹtradispersnyhčasticrm25 AT mankod dvusloinaâneironnaâsetʹobratnogorasprostraneniânaosnoveograničennyhmašinbolʹcmanadlâprognozirovaniâsutočnoikoncentraciiulʹtradispersnyhčasticrm25 |