Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері
In this paper, the new class of deep learning (DL) neural networks is considered and investigated — so-called hybrid DL networks based on self-organization method Group Method of Data Handling (GDMH). The application of GMDH enables not only to train neural weights, but also to construct the network...
Saved in:
| Date: | 2022 |
|---|---|
| Main Authors: | , , |
| Format: | Article |
| Language: | English |
| Published: |
The National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"
2022
|
| Subjects: | |
| Online Access: | https://journal.iasa.kpi.ua/article/view/259162 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Journal Title: | System research and information technologies |
| Download file: |  |
Institution
System research and information technologies| _version_ | 1867334426283737088 |
|---|---|
| author | Zaychenko, Yuriy Zaychenko, Helen Hamidov, Galib |
| author_facet | Zaychenko, Yuriy Zaychenko, Helen Hamidov, Galib |
| author_institution_txt_mv | [
{
"author": "Yuriy Zaychenko",
"institution": "Educational and Scientific Complex \"Institute for Applied System Analysis\" of the National Technical University of Ukraine \"Igor Sikorsky Kyiv Polytechnic Institute\", Kyiv"
},
{
"author": "Helen Zaychenko",
"institution": "Educational and Scientific Complex \"Institute for Applied System Analysis\" of the National Technical University of Ukraine \"Igor Sikorsky Kyiv Polytechnic Institute\", Kyiv"
},
{
"author": "Galib Hamidov",
"institution": "Azerishiq, Baku"
}
] |
| author_sort | Zaychenko, Yuriy |
| baseUrl_str | http://journal.iasa.kpi.ua/oai |
| collection | OJS |
| datestamp_date | 2022-06-21T10:27:50Z |
| description | In this paper, the new class of deep learning (DL) neural networks is considered and investigated — so-called hybrid DL networks based on self-organization method Group Method of Data Handling (GDMH). The application of GMDH enables not only to train neural weights, but also to construct the network structure as well. Different elementary neurons with two inputs may be used as nodes of this structure. So the advantage of such a structure is the small number of tuning parameters. In this paper, the optimization of parameters and the structure of hybrid neo-fuzzy networks was performed. The application of hybrid Dl networks for forecasting market indices was considered with various forecasting intervals: one day, one week, and one month. The experimental investigations of hybrid GMDH neo-fuzzy networks were carried out and comparison of its efficiency with FNN ANFIS in the forecasting problem was performed which enabled to estimate their efficiency and advantages. |
| doi_str_mv | 10.20535/SRIT.2308-8893.2022.1.06 |
| first_indexed | 2025-07-17T10:27:52Z |
| format | Article |
| fulltext |
Yu. Zaychenko, He. Zaichenko, G. Hamidov, 2022
Системні дослідження та інформаційні технології, 2022, № 1 73
TIДC
ТЕОРЕТИЧНІ ТА ПРИКЛАДНІ ПРОБЛЕМИ
ІНТЕЛЕКТУАЛЬНИХ СИСТЕМ ПІДТРИМАННЯ
ПРИЙНЯТТЯ РІШЕНЬ
UDC 519.925.51
DOI: 10.20535/SRIT.2308-8893.2022.1.06
HYBRID GMDH DEEP LEARNING NETWORKS – ANALYSIS,
OPTIMIZATION AND APPLICATIONS IN FORECASTING
AT FINANCIAL SPHERE
Yu. ZAYCHENKO, He. ZAICHENKO, G. HAMIDOV
Abstract. In this paper, the new class of deep learning (DL) neural networks is con-
sidered and investigated — so-called hybrid DL networks based on self-organization
method Group Method of Data Handling (GDMH). The application of GMDH en-
ables not only to train neural weights, but also to construct the network structure as
well. Different elementary neurons with two inputs may be used as nodes of this
structure. So the advantage of such a structure is the small number of tuning pa-
rameters. In this paper, the optimization of parameters and the structure of hybrid
neo-fuzzy networks was performed. The application of hybrid Dl networks for fore-
casting market indices was considered with various forecasting intervals: one day,
one week, and one month. The experimental investigations of hybrid GMDH neo-
fuzzy networks were carried out and comparison of its efficiency with FNN ANFIS
in the forecasting problem was performed which enabled to estimate their efficiency
and advantages.
Keywords: hybrid deep learning networks, self-organization, parameters and struc-
ture optimization, forecasting.
INTRODUCTION
Nowadays deep learning (DL) networks are widely used in different problems of
artificial intelligence: forecasting, pattern recognition, medical diagnostics,
etc.[1–4]. For its training various algorithms were developed usually based on
Back propagation method. Presence of many layers when using gradient algo-
rithm usually leads to occurrence drawbacks as vanishing or explosion of gradi-
ent. Therefore, the approach was suggested how to exclude this drawback to per-
form layer after layer training using stacked encoder-decoder or stacked restricted
Boltzmann machines [1, 2]. However, the problem is left how to choose the num-
ber of layers in DL network. The existing DL methods don’t enable to generate
structure of DL networks. But the training process will be more efficient if to
adapt not only neuron weights but the structure of network as well. For this goal
the application of GMDH method seems very promising. GMDH is based on
principle of self- organization and enables to construct network structure auto-
matically in the process of algorithm run [5–7]. In the previous years GMDH-
neural networks having active neurons [5–7], R-neurons [19], Q-neurons [3] as
Yu. Zaychenko, He. Zaichenko, G. Hamidov
ISSN 1681–6048 System Research & Information Technologies, 2022, № 1 74
nodes were developed; in the area integrating fuzzy GMDH and neural networks
the GMDH neuro-fuzzy and GMDH neo-fuzzy systems [13] were developed.
The very important property of GMDH is that as building blocks for con-
struction of a structure of DL networks elementary models with only two inputs,
so-called partial descriptions, are used. This allows to cut substantially training
time for hybrid Dl network as compared with conventional DL networks.
Therefore, new generation of deep learning — GMDH-hybrid neuro-fuzzy
networks were developed in [16] that combine advantages of the traditional
GMDH and DL fuzzy networks and may be trained with simple learning
procedures. The nodes of this network are Wang-Mendel elementary neural
networks with only two inputs. The experimental investigations of this class of
hybrid DL networks have shown their efficiency and preference over
conventional DL networks. But the drawbacks of application of Wang-Mendel
networks as nodes of hybrid DL networks lies herein that it’s necessary to train
not only neural weights but membership functions as well.
Later another class of hybrid DL networks — GMDH neo-fuzzy networks
were developed wherein as nodes of network neo-fuzzy neurons with two inputs
are used [17]. For their training its necessary to adapt only neuron weights that
demands less computational resources and cuts training time. That’s very impor-
tant for DL networks with a large number of hidden layers. The experimental
investigations of hybrid neo-fuzzy networks and comparison with conventional
DL network have shown their efficiency and less computational calculations for
training. But the problem is left to find the optimal parameters and structure of
hybrid neo- fuzzy networks and investigate them in practical applications.
The goal of this paper is to find optimal parameters and structure of hybrid
deep learning networks and investigate their efficiency in forecasting problem at
financial markets.
HYBRID NETWORK STRUCTURE OPTIMIZATION BASED ON GMDH
METHOD
The GMDH method was used to synthesize the structure of the hybrid network
based on the principle of self-organization. The principal idea of generation opti-
mal structure is the successive increase in the number of layers until the value of
the external criterion of optimality MSE begins to increase for the best model of
the current layer. In this case it is necessary to return to the previous layer, to find
there the best model with the minimum value of criterion. Then moving back-
ward, go through its connections, find the corresponding neurons of the previous
layer. This process continues until we reach the first layer and the corresponding
structure is automatically determined.
The process of synthesis of the network structure in the forward direction is
shown in Fig. 1 where in grey color the outputs which passed through selection
block (SB)are shown while in black color -outputs which were dropped (ex-
cluded) by SB.
The process of restoring the desired structure in the backward direction is
shown in Fig. 2. In the grey color nodes and their connections selected by this
process are indicated.
The corresponding optimal constructed structure of the hybrid network for
this forecasting problem is shown in Fig. 3.
Hybrid GMDH deep learning networks – analysis,optimization and applications …
Системні дослідження та інформаційні технології, 2022, № 1 75
F
ig
. 1
.
H
yb
ri
d
ne
tw
or
k
st
ru
ct
ur
e
co
ns
tr
uc
tio
n
us
in
g
G
M
D
H
m
et
ho
d
S
B
1
S
B
2
S
B
3
S
B
4
x 1
x 2
x 3
x 4
Yu. Zaychenko, He. Zaichenko, G. Hamidov
ISSN 1681–6048 System Research & Information Technologies, 2022, № 1 76
It consists of 3 layers: first layer has 3 neo-fuzzy neurons, second layer-
two neurons and the last- one neuron.
EXPERIMENTAL INVESTIGATIONS FOR SEARCH OPTIMAL PARAMETERS
OF HYBRID GMDH NEO-FUZZY NETWORK
The experimental investigations of hybrid GMDH neo-fuzzy network were per-
formed in the problem of Dow Jones and Nasdaq Index forecasting and compared
with FNN ANFIS. In the process of experiments optimal parameters and struc-
ture of hybrid GMDH networks were found. The experiments were performed
with different forecasting intervals: one day, one week and one month. For each
forecasting interval optimal parameters of hybrid neo-fuzzy networks were found
and investigated.
x1
x2
x3
x4
Fig. 2. Process of restoring found optimal structure in backward direction
x4
x3
x2
x1
Fig. 3. Optimal Structure of hybrid network for covid forecast constructed by GMDH
Hybrid GMDH deep learning networks – analysis,optimization and applications …
Системні дослідження та інформаційні технології, 2022, № 1 77
The Dow Jones is the stock index of the 30 largest American companies,
which was founded in 1896. The initial data was taken from Yahoo, a leading
financial information provider owned by Yahoo!
To prepare the initial data, data were uploaded at various intervals, namely
the value of the stock index by days, weeks and months. Each of the sets contains
the following data:
Date — data period;
Open — opening price;
High — the highest price for the period;
Low — the lowest price for the period;
Close — the price at the end of the period;
Adj Close — average closing price;
Volume — sales for the period.
The data set for the interval of one day contains 4867 records, of which non-
zero records are 4788 ones. The data set for the interval one month contains 1001
records, of which 1000 records are non-zero. The data set for the interval of one
month contains 195 records, of which 195 are non-zero.
Data normalizing. Reduction to a single scale is provided by normalization
of each variable to the range of its values. In the simplest case, it is a linear trans-
formation
min
max min
i
i
i i
x x
x
x x
m
, in the interval ]1,0[ix .
To find the most informative features as an input vector the network was al-
ternately trained on data sets that transmit only the following features subsets:
('Open', 'High', 'Low', 'Volume', 'Close'); ('Open', 'High', 'Low', 'Volume');
('Open', 'High', 'Low', 'Close'); ('Open', 'High', 'Low'); ('Open', 'High',
'Close');
('Open', 'High', 'Volume'); ('Open', 'Close', 'Low'); ('Open', 'Volume', 'Low');
('High', 'Low', 'Close'); ('Open', 'High'); ('High', 'Close'); ('Low', 'Close');
('Open', 'Volume').
The main network parameters that can be configured and are to be optimized
include the size of the input vector, the number of rules, and the function that sets
them, the number of parameters that are transferred to the next layer.
The size of the input vector is determined by the number of informative fea-
tures that are transmitted for training, and the number of days on the basis of
which the network gives the predicted value. Also, the number of network func-
tions that can be set includes the number of membership functions and their ap-
pearance, as well as the degree of freedom of choice of the system.
To select these parameters, it is necessary to conduct an experiment, training
the system, setting these parameters in the interval, and keeping those that give
the best results in the test sample.
The following parameters were investigated:
n — number of preceding days, based on which the forecasting is per-
formed (sliding window size) ]6;1[N ;
Yu. Zaychenko, He. Zaichenko, G. Hamidov
ISSN 1681–6048 System Research & Information Technologies, 2022, № 1 78
h — number of membership functions in each node, ]9;2[N ;
s — membership function parameter, where )1(
)(
hs
h
ab
;
b — an interval end;
a — an interval beginning;
h — membership functions number, which cover the interval;
;]5,1 ;01,0[s
f — number of parameters which are transferred to the network next
layer (freedom of choice).
The set of initial data was divided into a training sample and test sample in
the ratio of 70% and 30%, respectively.
Having launched GMDH neo-fuzzy system for training, values of MAE and
MAPE criteria were obtained with different combinations of these parameters.
For the Dow Jones stock index with different forecast intervals, the best
parameters for the different set of informative features were obtained as a result of
training and testing, which are shown in Table 1.
T a b l e 1 . The results of the selection of the optimal parameters of GMDH-neo-
fuzzy system for Dow Jones index with different prediction intervals
1 month 1 week Sets of informative
features n h f s МАЕ МАPE n h f s МАЕ МАPE
'Open', 'High', 'Low',
'Volume', 'Close'
1 2 2 1,0 0,0147 0,0452 2 4 2 0,7 0,0077 0,0295
'Open', 'High', 'Low',
'Volume'
1 2 3 1,3 0,0156 0,0476 2 4 3 0,9 0,0086 0,0332
'Open', 'High', 'Low',
'Close'
1 2 2 1,0 0,0147 0,0453 2 4 2 0,7 0,0077 0,0295
'Open', 'High', 'Low' 1 2 3 1,3 0,0156 0,0476 2 4 3 0,9 0,0086 0,0332
'Open', 'High', '
Close'
1 2 3 1,2 0,0153 0,0467 2 4 3 0,9 0,0079 0,0309
'Open', 'High', '
Volume'
5 2 5 0,1 0,0177 0,0654 2 4 3 1,0 0,0098 0,0380
'Open', 'Low', 'Close' 1 2 3 1,2 0,0147 0,0456 2 4 3 0,7 0,0081 0,0308
'Open', 'Volume', '
Low'
5 3 7 0,1 0,0171 0,0644 4 2 6 0,1 0,0095 0,0348
'High', 'Low', 'Close' 1 2 2 1,0 0,0147 0,0453 2 4 2 0,7 0,0077 0,0295
'Open', 'High' 5 2 5 0,1 0,0177 0,0654 2 4 3 1,0 0,0098 0,0380
'Open', 'Close' 1 2 2 1,3 0,0165 0,0498 2 4 3 0,6 0,0085 0,0331
'High', 'Close' 1 2 2 1,2 0,0154 0,0467 2 4 3 0,9 0,0079 0,0309
'Low', 'Close' 1 2 2 1,2 0,0147 0,0456 2 4 2 0,7 0,0081 0,0306
'Open', 'Volume' 5 2 2 0,8 0,0189 0,0689 3 4 2 0,1 0,0112 0,0445
Thus, analyzing presented results one may conclude that the most informa-
tive for GMDH neo-fuzzy system are the following sets of features: ['Open',
'High', 'Close'], ['Open', 'Low', 'Close'], ['High', 'Low', 'Close'], ['High', 'Close'],
['Low', 'Close'].
Hybrid GMDH deep learning networks – analysis,optimization and applications …
Системні дослідження та інформаційні технології, 2022, № 1 79
For the Dow Jones stock index for one month forecast period, the following
optimal configurations of GMDH neo-fuzzy network were obtained:
the number of informative features — 3;
the number of periods on the basis of which the forecast is made — 1;
the number of membership functions in each of the nodes — 2;
the number of layers — 2;
the number of nodes in the first layer — 3;
number of nodes on the second layer — 1.
For the Dow Jones stock index for the one week forecast period, the follow-
ing optimal configurations of the GMDH neo-fuzzy system were obtained:
the number of informative features — 3;
the number of periods on the basis of which the forecast is made — 2;
the number of membership functions in each of the nodes — 4;
the number of layers — 2;
the number of nodes on the first layer — 24;
the number of nodes on the second layer — 1.
The form of the membership function for forecasting interval of one week is
shown in the Fig. 4.
For the Dow Jones stock index for one day forecast period, the following
optimal configurations of GMDH neo-fuzzy network were obtained:
number of informative features — 3;
the number of periods on the basis of which the forecast is made — 5;
the number of membership functions in each of the nodes — 2;
the number of layers — 2;
the number of nodes in the first layer — 30;
the number of nodes in the second layer — 1.
In the next series of experiments the optimal parameters of hybrid GMDH
neo-fuzzy network were searched for the problem of Nasdaq index forecast with
different forecasting intervals. The optimal parameters and sets of informative
features for interval one month and one week are presented in the Table 2, while
for the interval one day — in the Table 3.
1,0
0,8
0,6
0,4
0,2
0,0
–1,0 –0,5 0,0 0,5 1,0 1,5 2,5
y
—
a
xi
s
x — axis
Fig. 4. Forms of the membership function of Dow Jones index for the forecast period
of 1 week
Yu. Zaychenko, He. Zaichenko, G. Hamidov
ISSN 1681–6048 System Research & Information Technologies, 2022, № 1 80
T a b l e 2 . The results of the selection of the optimal parameters of GMDH
neo-fuzzy system for Nasdaq index with different prediction intervals
1 month 1 week Sets of informative
features n h f s МАЕ МАPE n h f s МАЕ МАPE
'Open', 'High', 'Low',
'Volume', 'Close'
1 3 2 0,58 0,0090 0,0796 3 3 2 0,1 0,0043 0,0400
'Open', 'High', 'Low',
'Volume'
1 3 3 0,78 0,0088 0,0812 5 2 3 0,7 0,0048 0,0445
'Open', 'High', 'Low',
'Close'
1 3 3 0,58 0,090 0,0796 3 3 2 0,1 0,0044 0,0400
'Open', 'High', 'Low' 1 3 3 0,78 0,0088 0,0812 5 2 3 0,7 0,0048 0,0445
'Open', 'High', 'Close' 1 3 3 0,68 0,0095 0,0824 3 3 2 0,1 0,0045 0,0427
'Open', 'High',
'Volume'
2 3 3 0,18 0,0109 0,1124 4 8 3 0,9 0,0054 0,0520
'Open', 'Low', 'Close' 1 3 3 0,88 0,0085 0,0850 3 5 2 0,5 0,0044 0,0414
'Open', 'Volume',
'Low'
2 3 3 0,48 0,0095 0,0941 5 2 4 0,1 0,0051 0,0472
'High', 'Low', 'Close' 2 5 3 0,08 0,0089 0,0796 3 3 2 0,1 0,0043 0,0400
'Open', 'High' 2 3 3 0,18 0,0109 0,1128 4 8 3 0,9 0,0054 0,0520
'Open', 'Close' 2 2 3 0,18 0,0093 0,1011 4 6 2 0,5 0,0046 0,0421
'High', 'Close' 1 3 2 0,68 0,0095 0,1066 3 3 2 0,1 0,0045 0,0427
'Low', 'Close' 1 3 2 0,88 0,0085 0,085 3 5 2 0,5 0,0044 0,0414
'Open', Volume' 2 5 4 1,38 0,0121 0,1503 4 7 3 0,9 0,0064 0,0597
T a b l e 3 . The results of the selection of the optimal parameters of GMDH
neo-fuzzy system for Nasdaq index with one day prediction interval
1 day
Sets of informative features
n h f s МАЕ МАPE
'Open', 'High', 'Low', 'Volume', 'Close' 6 8 2 0,1 0,0023 0,0193
'Open', 'High', 'Low', 'Volume' 6 7 3 0,7 0,0026 0,0232
'Open', 'High', 'Low', 'Close' 6 8 2 0,1 0,0023 0,0193
'Open', 'High', 'Low' 6 7 3 0,7 0,0026 0,0232
'Open', 'High', 'Close' 6 7 2 0,1 0,0024 0,0204
'Open', 'High', 'Volume' 6 10 3 0,1 0,0030 0,0262
'Open', 'Low', 'Close' 6 7 5 0,1 0,0024 0,0200
'Open', 'Volume', 'Low' 6 9 5 0,1 0,0028 0,0242
'High', 'Low', 'Close' 6 8 2 0,1 0,0023 0,0193
'Open', 'High' 1 7 2 0,1 0,0029 0,0241
'Open', 'Close' 6 8 6 0,1 0,0025 0,0213
'High', 'Close' 6 7 2 0,1 0,0024 0,0205
'Low', 'Close' 6 9 6 0,1 0,0024 0,0202
'Open', 'Volume' 6 7 2 0,1 0,0034 0,0288
For Nasdaq stock index for one month forecast period, the following optimal
configurations of GMDH-neo-fuzzy network were obtained:
the number of informative features — 4;
the number of periods on the basis of which the forecast is made — 1;
Hybrid GMDH deep learning networks – analysis,optimization and applications …
Системні дослідження та інформаційні технології, 2022, № 1 81
the number of membership functions in each of the nodes — 3;
the number of layers — 2;
the number of nodes in the first layer — 12;
number of nodes on the second layer — 1.
For Nasdaq stock index for the one week forecast period, the following op-
timal configurations of the GMDH-neo-fuzzy system were obtained:
the number of informative features — 4;
the number of periods on the basis of which the forecast is made — 3;
the number of membership functions in each of the nodes — 3;
the number of layers — 2;
the number of nodes on the first layer — 36;
the number of nodes on the second layer — 1.
In the Fig. 5 forms of membership functions of Nasdaq index for one month
forecast are presented.
For Nasdaq index with forecasting interval 1 month the following results
were obtained:
MAE — 0,02812;
MAPE — 0,03165;
Forecasting time — 0,0005815 s.
For Nasdaq index with forecasting interval 1 week the following results were
obtained:
MAE — 0,0099397;
MAPE — 0,0109336;
Forecasting time — 0,0003004 s.
For Nasdaq index with forecasting interval 1 day the following results were
obtained
MAE — 0,005740;
MAPE — 0,0063267;
Forecasting time — 0,000287 s.
Next, experiments were performed to find the optimal values of the parame-
ters of FNN ANFIS. The size of the input vector is determined by the number of
informative features that are transmitted for training, and the number of days of
prehistory, on the basis of which the forecasting is performed.
Fig. 5. Forms of the membership function of Nasdaq index for the forecast
period of 1 month
1,0
0,8
0,6
0,4
0,2
0,0
–1,0 –0,5 0,0 0,5 1,0 1,5 2,5
y
—
a
xi
s
x — axis
Yu. Zaychenko, He. Zaichenko, G. Hamidov
ISSN 1681–6048 System Research & Information Technologies, 2022, № 1 82
To select these parameters, an experiment was performed, including training
of the network, setting these parameters in the interval, and choosing those that
give the best results at the test sample.
The set of initial data was divided into a training sample and test data in the
proportion of 70% and 30%, respectively. By launching the ANFIS network with
different combinations of these parameters, data on MAE and MAPE criteria
were obtained.
For the Dow Jones stock index one month forecast period, the following op-
timal ANFIS network configurations were obtained:
number of informative features — 3;
number of nodes — 6;
the number of periods on the basis of which the forecast is made — 2;
the number of membership functions in each of the nodes — 6.
The optimal parameters of FNN ANFIS for Dow Jones index forecast are
shown in Table 4.
T a b l e 4 . The results of the selection of the optimal characteristics of ANFIS
network for Dow Jones index with different forecast intervals
1 month 1 week 1 day Sets of informative
features n h МАЕ МАPE n h МАЕ МАPE n h МАЕ МАPE
'Open', 'High', 'Low' 2 6 0,222 0,0710 1 9 0,0091 0,0334 1 10 0,0037 0,0142
'Open', 'High', 'Close' 2 3 0,0223 0,0727 2 8 0,0080 0,0303 1 11 0,0034 0,0129
'Open', 'Low', 'Close' 2 6 0,0192 0,0680 2 10 0,0804 0,0307 1 5 0,0045 0,0154
'High', 'Low', 'Close' 2 8 0,0209 0,0720 2 9 0,0903 0,0325 2 10 0,0036 0,0134
'High', 'Close' 2 9 0,0223 0,0750 1 3 0,0077 0,0282 1 7 0,0035 0,0135
'Low', 'Close' 2 7 0,0201 0,0691 1 5 0,0094 0,0338 1 5 0,0035 0,0136
After finding all the optimal parameters of GMDH neo-fuzzy system and
training parameters, the system was trained, and then the data for prediction was
provided. Training and testing of the system took place on data for the period up
to 01.01.2021 for monthly periods, and until 01.06.2021 for weekly and day peri-
ods. Forecasting was
based on data for the pe-
riod after 01.01.2021 for
monthly periods and
after 01.06.2021 for day
and week periods. For
Dow Jones index with a
forecast period of one
month, the following
forecasting data were
obtained: MAE —
0,02952; MAPE —
0,0335; forecasting time
— 0,00025
Learning and fore-
casting results are shown
in Fig. 6.
0,10
0,05
0,00
0,8
0,6
0,4
0,2
Fig. 6. Results of training and forecasting Dow Jones
Index with interval one month by hybrid GMDH neo-
fuzzy system
Hybrid GMDH deep learning networks – analysis,optimization and applications …
Системні дослідження та інформаційні технології, 2022, № 1 83
COMPARISON OF FORECASTING RESULTS OF GMDH NEO-FUZZY
SYSTEM AND ANFIS NETWORK
Experimental investigations of the accuracy of market indexes Dow Jones and
Nasdaq forecasting with forecasting intervals of one month, one week and one
day were performed, using a hybrid GMDH neo-fuzzy network. For each predic-
tion interval the optimal parameters found in previous experiments were selected.
A comparative analysis with the forecasting results obtained by FNN ANFIS was
performed.
According to the results of forecasting, values of MAE, MAPE and training
time for each type of neural network were obtained. All comparison results are
summarized in Tables 5–7 for Dow Jones index and in Tables 8–10 for Nasdaq
index.
T a b l e 5 . Comparison of the forecasting results of GMDH neo-fuzzy neural
network and FNN ANFIS for Dow Jones Index with forecasting interval 1 month
Criterion
GMDH neo-fuzzy
neural network
FNN ANFIS Difference
MAE at training sample 0,016938 0,016135 4,70%
MAPE at training sample 0,061866 0,052607 14,97%
MAE at test sample 0,02952 0,096734 -227,68%
MAPE at test sample 0,03350 0,107397 -220,59%
Training time (sec) 0,0023246 75,258 32375x
Forecasting time (sec) 0,0003123 0,02652 84,92x
T a b l e 6 . Comparison of the forecasting results of GMDH neo-fuzzy neural
network and FNN ANFIS for Dow Jones Index with forecasting interval 1 week
Criterion
GMDH neo-fuzzy
neural network
FNN ANFIS Difference
MAE at training sample 0,007949 0,008564 -7,74%
MAPE at training sample 0,029890 0,029291 2,00%
MAE at test sample 0,011476 0,019279 -67,99%
MAPE at test sample 0,012468 0,020923 -67,82%
Training time (sec) 0,012840 194,3520 14980x
Forecasting time (sec) 0,00027132 0,028604 105,42x
T a b l e 7 . Comparison of the forecasting results of GMDH neo-fuzzy neural
network and FNN ANFIS for Dow Jones Index with forecasting interval 1 day
Criterion
GMDH neo-fuzzy
neural network
FNN ANFIS Difference
MAE at training sample 0,003618 0,004234 -17,03%
MAPE at training sample 0,013981 0,014067 -0,615%
MAE at test sample 0,005348 0,005822 -8,86%
MAPE at test sample 0,005812 0,005822 -0,172%
Training time (sec) 0,19944 876,3658 4394,13x
Forecasting time (sec) 0,00040317 0,038055 94,39x
Yu. Zaychenko, He. Zaichenko, G. Hamidov
ISSN 1681–6048 System Research & Information Technologies, 2022, № 1 84
T a b l e 8 . Comparison of the forecasting results of GMDH neo-fuzzy neural
network and FNN ANFIS for Nasdaq Index with forecasting interval 1 month
Criterion GMDH neo-fuzzy
neural network FNN ANFIS Difference
MAE at training sample 0,011264 0,011140 1,10%
MAPE at training sample 0,098307 0,088272 10,21%
MAE at test sample 0,006635 0,008617 -59,87%
MAPE at test sample 0,060995 0,097332 -59,57%
Training time (sec) 0,0065255 34,5328 5291,9x
Forecasting time (sec) 0,0005815 0,024286 41,76x
T a b l e 9 . Comparison of the forecasting results of GMDH neo-fuzzy neural
network and FNN ANFIS for Nasdaq Index with forecasting interval 1 week
Criterion GMDH neo-fuzzy
neural network FNN ANFIS Difference
MAE at training sample 0,0052929 0,0055274 -4,43%
MAPE at training sample 0,041831 0,052723 -26,04%
MAE at test sample 0,009940 0,012973 -30,51%
MAPE at test sample 0,010933 0,014203 -29,91%
Training time (sec) 0,0411811 175,5418 4262,7x
Forecasting time (sec) 0,00030041 0,02489 82,85x
T a b l e 1 0 . Comparison of the forecasting results of GMDH neo-fuzzy neural
network and FNN ANFIS for Nasdaq Index with forecasting interval 1 day
Criterion GMDH neo-fuzzy
neural network FNN ANFIS Difference
MAE at training sample 0,002349 0,002798 -19,11%
MAPE at training sample 0,019121 0,025317 -32,40%
MAE at test sample 0,005740 0,007161 -24,76%
MAPE at test sample 0,0063267 0,0079001 -24,87%
Training time (sec) 3,8612 823,90 213,39x
Forecasting time (sec) 0,0004616 0,085263 184,72x
Analyzing the presented results one may conclude, the best forecasting re-
sults for all forecasting intervals were obtained for hybrid GMDH neo-fuzzy sys-
tem for both indexes Dow Jones and Nasdaq. The worst forecasting result for
ANFIS network was obtained for one month forecasting period. The largest dif-
ference in the accuracy of forecasting by both criteria was obtained for the fore-
casting period of one month (over 200%). As the forecasting period decreases, the
gap between the networks accuracy also decreases.
In addition, training and direct prediction times were also significantly less
for hybrid GMDH neo-fuzzy system as compared with ANFIS.
CONCLUSION
In the paper new generation of Deep learning networks-hybrid GMDH neo-fuzzy
networks are considered, optimized and investigated.
The algorithm of hybrid network structure synthesis is presented and demon-
strated at the problem of forecasting.
The experimental investigations of the hybrid networks were carried out and
compared with conventional DL networks. The problem of forecasting Dow Jones
Hybrid GMDH deep learning networks – analysis,optimization and applications …
Системні дослідження та інформаційні технології, 2022, № 1 85
and Nasdaq Index with application of hybrid neo-fuzzy networks was considered,
investigated and compared with FNN ANFIS at the different forecasting intervals:
one month, one week and day.
The optimal parameters of hybrid neo-fuzzy networks and sets of informa-
tive features for forecasting problems were found. The experimental results have
shown the forecasting accuracy of hybrid neo-fuzzy networks is much better than
for FNN ANFIS.
The training time is the least for hybrid neo-fuzzy network as compared with
alternative ANFIS network.
In a whole the hybrid DL networks based on GMDH are free from draw-
backs of conventional DL networks- decay or explosion of gradient. Besides, they
enable to construct optimal network structure automatically in the process of algo-
rithm GMDH run and additionally they demand less computational costs for train-
ing due to small number of tunable parameters (only two) in every hidden node as
compared with DL networks of general structure. That’s is especially significant
for DL networks with large number of layers.
REFERENCES
1. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
2. G. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets”,
Neural Computation, vol. 18, no. 7, pp. 1527–1554, May 2006.
3. Y. Bengio, Y. LeCun, and G. Hinton, “Deep learning”, Nature, no. 521, pp. 436–444, May 2015.
4. J. Schmidhuber, “Deep learning in neural networks: an overview”, Neural Networks,
no. 61, pp. 85–117, 2015.
5. A.G. Ivakhnenko, G.A. Ivakhnenko, and J.A. Mueller, “Self-organization of the neural
networks with active neurons”, Pattern Recognition and Image Analysis, 4, 2,
pp. 177–188, 1994.
6. A.G. Ivakhnenko, D. Wuensch, and G.A. Ivakhnenko, “Inductive sorting-out GMDH al-
gorithms with polynomial complexity for active neurons of neural networks”, Neural
Networks, 2, pp. 1169–1173, 1999.
7. G.A. Ivakhnenko, “Self-organization of neuronet with active neurons for effects of
nuclear test explosions forecasting”, System Analysis Modeling Simulation, 20,
pp. 107–116, 1995.
8. M. Zgurovsky and Yu. Zaychenko, Fundamentals of computational intelligence: System
approach. Springer, 2016.
9. L.-X. Wang and J.M. Mendel, “Fuzzy basis functions, universal approximation, and or-
thogonal least-squares learning”, IEEE Trans. on Neural Networks, vol. 3, no. 5,
pp. 807–814, 1992.
10. J.-S. Jang, “ANFIS: Adaptive-network-based fuzzy inference systems”, IEEE Trans. on
Systems, Man, and Cybernetics, 23, pp. 665–685, 1993.
11. T. Yamakawa, E. Uchino, T. Miki, and H. Kusanagi, “A neo-fuzzy neuron and its appli-
cations to system identification and prediction of the system behavior”, in Proc. 2nd
Intеrn. Conf. Fuzzy Logic and Neural Networks «LIZUKA-92», Lizuka, 1992, pp. 477–483.
12. Ye. Bodyanskiy, N. Teslenko, and P. Grimm, “Hybrid evolving neural network using
kernel activation functions”, in Proc. 17th Zittau East-West Fuzzy Colloquium, Zit-
tau/Goerlitz, HS, 2010, pp. 39–46.
13. Ye. Bodyanskiy, Yu. Zaychenko, E. Pavlikovskaya, M. Samarina, and Ye. Viktorov,
“The neo-fuzzy neural network structure optimization using the GMDH for the solv-
ing forecasting and classification problems”, Proc. Int. Workshop on Inductive Modeling,
Krynica, Poland, 2009, pp. 77–89.
14. Ye. Bodyanskiy, O. Vynokurova, A. Dolotov, and O. Kharchenko, “Wavelet-neuro-fuzzy
network structure optimization using GMDH for the solving forecasting tasks”, in
Proc. 4th Int. Conf. on Inductive Modelling ICIM 2013, Kyiv, 2013, pp. 61–67.
15. Ye. Bodyanskiy, O. Vynokurova, and N. Teslenko, “Cascade GMDH-wavelet-neuro-
fuzzy network”, in Proc. 4th Int. Workshop on Inductive Modeling “IWIM 2011”, Kyiv,
Ukraine, 2011, pp. 22–30.
Yu. Zaychenko, He. Zaichenko, G. Hamidov
ISSN 1681–6048 System Research & Information Technologies, 2022, № 1 86
16. Ye. Bodyanskiy, O. Boiko Yu. Zaychenko, and G. Hamidov, “Evolving Hybrid GMDH-
Neuro-Fuzzy Network and Its Applications”, in Proceedings of the International confer-
ence SAIC 2018, Kiev, Ukraine, 2018.
17. Evgeniy Bodyanskiy, Yuriy Zaychenko, Olena Boiko, Galib Hamidov, and Anna Ze-
likman, “The hybrid GMDH-neo-fuzzy neural network in forecasting problems in finan-
cial sphere”, in Proceedings of the International conference IEEE SAIC 2020, Kiev,
Ukraine, 2020.
18. T. Ohtani, “Automatic variable selection in RBF network and its application to neuro-
fuzzy GMDH”, Proc. Fourth Int. Conf. on Knowledge-Based Intelligent Engineering
Systems and Allied Technologies, 2000, vol. 2, pp. 840–843.
Received 17.01.2022
INFORMATION ON THE ARTICLE
Yuriy P. Zaychenko, ORCID: 0000-0001-9662-3269, Institute for Applied System
Analysis of the National Technical University of Ukraine “Igor Sikorsky Kyiv Polytech-
nic Institute”, Ukraine, e-mail: zaychenkoyuri@ukr.net
Helen Yu. Zaychenko, ORCID: 0000-0002-4546-0428, Institute for Applied System
Analysis of the National Technical University of Ukraine “Igor Sikorsky Kyiv Polytech-
nic Institute”, Ukraine, e-mail: syncmaster@bigmir.net
Galib Hamidov, “Azerishiq”, Azerbaijan, e-mail: galib.hamidov@gmail.com
ГИБРИДНІ МГУА-МЕРЕЖІ ГЛИБОКОГО НАВЧАННЯ — АНАЛІЗ,
ОПТИМІЗАЦИЯ ТА ЗАСТОСУВАННЯ ДЛЯ ПРОГНОЗУВАННЯ У
ФІНАНСОВІЙ СФЕРІ / Ю.П. Зайченко, О.Ю. Зайченко, Г. Гамідов
Анотація. Розглянуто та досліджено новий клас мереж глибокого навчання —
гібридні мережі глибокого навчання на основі методу самоорганізації МГУА.
Застосування МГУА дозволяє навчати не тільки ваги зв’язків, але і конструю-
вати структуру мережі. Як вузли мережі можуть бути використані елементарні
нейрони з двома входами. Перевага такої структури — мала кількість налаш-
товуваних параметрів. Виконано оптимізацію параметрів та структури
гібридних неофаззі мереж. Розглянуто застосування гібридних мереж глибоко-
го навчання з оптимізованими параметрами для прогнозування біржових
індексів з різними інтервалами упередження — один день, тиждень та місяць.
Проведено експериментальні дослідження гібридних МГУА неофаззі мереж та
порівняння їх з нечіткою нейронною мережею ANFIS, що дозволило оцінити
ефективність та переваги гібридних мереж порівняно звичайними мережами
глибокого навчання.
Ключові слова: гібридні мережі глибокого навчання, самоорганізація, оптимі-
зація параметрів і структури, прогнозування.
ГИБРИДНЫЕ МГУА-СЕТИ ГЛУБОКОГО ОБУЧЕНИЯ — АНАЛИЗ,
ОПТИМИЗАЦИЯ И ПРИМЕНЕНИЯ ДЛЯ ПРОГНОЗИРОВАНИЯ В
ФИНАНСОВОЙ СФЕРЕ / Ю.П. Зайченко, Е.Ю. Зайченко, Г. Гамидов
Аннотация. Рассмотрен и исследован новый класс сетей глубокого обучения —
гибридные сети глубокого обучения на основе метода самоорганизации
МГУА. Применение МГУА позволяет обучать не только веса связей, но и
конструировать структуру сети. В качестве узлов сети могут быть исполь-
зованы элементарные нейроны с двумя входами. Преимущество такой струк-
туры — малое количество настраиваемых параметров. Выполнена оптимиза-
ция параметров и структуры гибридных неофаззи сетей. Рассмотрено
применение гибридных сетей глубокого обучения с оптимизированными па-
раметрами для прогнозирования биржевых индексов с различными интервала-
ми упреждения — один день, неделя и месяц. Проведены экспериментальные ис-
следования гибридных МГУА неофаззи сетей и сравнение их с нечеткой
нейронной сетью ANFIS, что позволило оценить эффективность и преимуще-
ства гибридных сетей по сравнению обычными сетями глубокого обучения.
Ключевые слова: гибридные сети глубокого обучения, самоорганизация, оп-
тимизация параметров и структуры, прогнозирование.
|
| id | journaliasakpiua-article-259162 |
| institution | System research and information technologies |
| keywords_txt_mv | keywords |
| language | English |
| last_indexed | 2025-07-17T10:27:52Z |
| publishDate | 2022 |
| publisher | The National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute" |
| record_format | ojs |
| resource_txt_mv | journaliasakpiua/75/dc2e438c17530df98489910384374475.pdf |
| spelling | journaliasakpiua-article-2591622022-06-21T10:27:50Z Hybrid GMDH deep learning networks – analysis, optimization and applications in forecasting at financial sphere Гибридные МГУА-сети глубокого обучения — анализ, оптимизация и применения для прогнозирования в финансовой сфере Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері Zaychenko, Yuriy Zaychenko, Helen Hamidov, Galib гібридні мережі глибокого навчання самоорганізація оптимізація параметрів і структури прогнозування гибридные сети глубокого обучения самоорганизация оптимизация параметров и структуры прогнозирование hybrid deep learning networks self-organization parameters and structure optimization forecasting In this paper, the new class of deep learning (DL) neural networks is considered and investigated — so-called hybrid DL networks based on self-organization method Group Method of Data Handling (GDMH). The application of GMDH enables not only to train neural weights, but also to construct the network structure as well. Different elementary neurons with two inputs may be used as nodes of this structure. So the advantage of such a structure is the small number of tuning parameters. In this paper, the optimization of parameters and the structure of hybrid neo-fuzzy networks was performed. The application of hybrid Dl networks for forecasting market indices was considered with various forecasting intervals: one day, one week, and one month. The experimental investigations of hybrid GMDH neo-fuzzy networks were carried out and comparison of its efficiency with FNN ANFIS in the forecasting problem was performed which enabled to estimate their efficiency and advantages. Рассмотрен и исследован новый класс сетей глубокого обучения — гибридные сети глубокого обучения на основе метода самоорганизации МГУА. Применение МГУА позволяет обучать не только веса связей, но и конструировать структуру сети. В качестве узлов сети могут быть использованы элементарные нейроны с двумя входами. Преимущество такой структуры — малое количество настраиваемых параметров. Выполнена оптимизация параметров и структуры гибридных неофаззи сетей. Рассмотрено применение гибридных сетей глубокого обучения с оптимизированными параметрами для прогнозирования биржевых индексов с различными интервалами упреждения — один день, неделя и месяц. Проведены экспериментальные исследования гибридных МГУА неофаззи сетей и сравнение их с нечеткой нейронной сетью ANFIS, что позволило оценить эффективность и преимущества гибридных сетей по сравнению обычными сетями глубокого обучения. Розглянуто та досліджено новий клас мереж глибокого навчання — гібридні мережі глибокого навчання на основі методу самоорганізації МГУА. Застосування МГУА дозволяє навчати не тільки ваги зв’язків, але і конструювати структуру мережі. Як вузли мережі можуть бути використані елементарні нейрони з двома входами. Перевага такої структури — мала кількість налаштовуваних параметрів. Виконано оптимізацію параметрів та структури гібридних неофаззі мереж. Розглянуто застосування гібридних мереж глибокого навчання з оптимізованими параметрами для прогнозування біржових індексів з різними інтервалами упередження — один день, тиждень та місяць. Проведено експериментальні дослідження гібридних МГУА неофаззі мереж та порівняння їх з нечіткою нейронною мережею ANFIS, що дозволило оцінити ефективність та переваги гібридних мереж порівняно звичайними мережами глибокого навчання. The National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute" 2022-04-25 Article Article application/pdf https://journal.iasa.kpi.ua/article/view/259162 10.20535/SRIT.2308-8893.2022.1.06 System research and information technologies; No. 1 (2022); 73-86 Системные исследования и информационные технологии; № 1 (2022); 73-86 Системні дослідження та інформаційні технології; № 1 (2022); 73-86 2308-8893 1681-6048 en https://journal.iasa.kpi.ua/article/view/259162/255820 |
| spellingShingle | гібридні мережі глибокого навчання самоорганізація оптимізація параметрів і структури прогнозування Zaychenko, Yuriy Zaychenko, Helen Hamidov, Galib Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері |
| title | Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері |
| title_alt | Hybrid GMDH deep learning networks – analysis, optimization and applications in forecasting at financial sphere Гибридные МГУА-сети глубокого обучения — анализ, оптимизация и применения для прогнозирования в финансовой сфере |
| title_full | Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері |
| title_fullStr | Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері |
| title_full_unstemmed | Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері |
| title_short | Гибридні МГУА-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері |
| title_sort | гибридні мгуа-мережі глибокого навчання — аналіз, оптимізация та застосування для прогнозування у фінансовій сфері |
| topic | гібридні мережі глибокого навчання самоорганізація оптимізація параметрів і структури прогнозування |
| topic_facet | гібридні мережі глибокого навчання самоорганізація оптимізація параметрів і структури прогнозування гибридные сети глубокого обучения самоорганизация оптимизация параметров и структуры прогнозирование hybrid deep learning networks self-organization parameters and structure optimization forecasting |
| url | https://journal.iasa.kpi.ua/article/view/259162 |
| work_keys_str_mv | AT zaychenkoyuriy hybridgmdhdeeplearningnetworksanalysisoptimizationandapplicationsinforecastingatfinancialsphere AT zaychenkohelen hybridgmdhdeeplearningnetworksanalysisoptimizationandapplicationsinforecastingatfinancialsphere AT hamidovgalib hybridgmdhdeeplearningnetworksanalysisoptimizationandapplicationsinforecastingatfinancialsphere AT zaychenkoyuriy gibridnyemguasetiglubokogoobučeniâanalizoptimizaciâiprimeneniâdlâprognozirovaniâvfinansovojsfere AT zaychenkohelen gibridnyemguasetiglubokogoobučeniâanalizoptimizaciâiprimeneniâdlâprognozirovaniâvfinansovojsfere AT hamidovgalib gibridnyemguasetiglubokogoobučeniâanalizoptimizaciâiprimeneniâdlâprognozirovaniâvfinansovojsfere AT zaychenkoyuriy gibridnímguamerežíglibokogonavčannâanalízoptimízaciâtazastosuvannâdlâprognozuvannâufínansovíjsferí AT zaychenkohelen gibridnímguamerežíglibokogonavčannâanalízoptimízaciâtazastosuvannâdlâprognozuvannâufínansovíjsferí AT hamidovgalib gibridnímguamerežíglibokogonavčannâanalízoptimízaciâtazastosuvannâdlâprognozuvannâufínansovíjsferí |