Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling
The purpose of this paper is analysing the background of the GMDH invention by Ivakhnenko and the evolution of model self-organization ideas, methods and tools during the halfcentury historical period of successful development of the inductive modeling methodology.
Saved in:
| Date: | 2018 |
|---|---|
| Main Author: | |
| Format: | Article |
| Language: | English |
| Published: |
Міжнародний науково-навчальний центр інформаційних технологій і систем НАН України та МОН України
2018
|
| Series: | Кибернетика и вычислительная техника |
| Subjects: | |
| Online Access: | https://nasplib.isofts.kiev.ua/handle/123456789/150497 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Journal Title: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Cite this: | Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling / V.S. Stepashko // Кибернетика и вычислительная техника. — 2018. — № 4 (194). — С. 41-60. — Бібліогр.: 80 назв. — англ. |
Institution
Digital Library of Periodicals of National Academy of Sciences of Ukraine| id |
nasplib_isofts_kiev_ua-123456789-150497 |
|---|---|
| record_format |
dspace |
| spelling |
nasplib_isofts_kiev_ua-123456789-1504972025-02-09T20:04:23Z Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling Формування i розвиток самоорганізовних інтелектуальних технологій індуктивного моделювання Формирование и развитие самоорганизующихся интеллектуальных технологий индуктивного моделирования Stepashko, V.S. Интеллектуальное управление и системы The purpose of this paper is analysing the background of the GMDH invention by Ivakhnenko and the evolution of model self-organization ideas, methods and tools during the halfcentury historical period of successful development of the inductive modeling methodology. Метою дослідження є аналіз передумов винайдення МГУА О.Г. Івахненком та еволюції ідей, методів та інструментів самоорганізації моделей протягом піввікового історичного періоду успішного розвитку методології індуктивного моделювання. Целью работы является анализ эволюции идей, методов и инструментов самоорганизации моделей в течение полувекового исторического периода успешного развития методологии индуктивного моделирования. Проанализированы основные предпосылки создания академиком А.Г. Ивахненко метода группового учета аргументов (МГУА), исследуется эволюция его научных идей и взглядов, а также основные достижения в развитии МГУА в период 1968–1997 годов. Охарактеризован вклад исследователей разных стран в модификацию и применение МГУА. Приведены результаты дальнейших разработок методов и инструментов индуктивного моделирования в отделе Информационных технологий индуктивного моделирования и указаны наиболее перспективные направления исследований в этой области. 2018 Article Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling / V.S. Stepashko // Кибернетика и вычислительная техника. — 2018. — № 4 (194). — С. 41-60. — Бібліогр.: 80 назв. — англ. 0454-9910 DOI: https:// 10.15407/kvt194.04.041 https://nasplib.isofts.kiev.ua/handle/123456789/150497 004.942: 681.513.8 en Кибернетика и вычислительная техника application/pdf Міжнародний науково-навчальний центр інформаційних технологій і систем НАН України та МОН України |
| institution |
Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| collection |
DSpace DC |
| language |
English |
| topic |
Интеллектуальное управление и системы Интеллектуальное управление и системы |
| spellingShingle |
Интеллектуальное управление и системы Интеллектуальное управление и системы Stepashko, V.S. Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling Кибернетика и вычислительная техника |
| description |
The purpose of this paper is analysing the background of the GMDH invention by Ivakhnenko and the evolution of model self-organization ideas, methods and tools during the halfcentury historical period of successful development of the inductive modeling methodology. |
| format |
Article |
| author |
Stepashko, V.S. |
| author_facet |
Stepashko, V.S. |
| author_sort |
Stepashko, V.S. |
| title |
Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling |
| title_short |
Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling |
| title_full |
Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling |
| title_fullStr |
Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling |
| title_full_unstemmed |
Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling |
| title_sort |
formation and development of self-organizing intelligent technologies of inductive modeling |
| publisher |
Міжнародний науково-навчальний центр інформаційних технологій і систем НАН України та МОН України |
| publishDate |
2018 |
| topic_facet |
Интеллектуальное управление и системы |
| url |
https://nasplib.isofts.kiev.ua/handle/123456789/150497 |
| citation_txt |
Formation and Development of Self-Organizing Intelligent Technologies of Inductive Modeling / V.S. Stepashko // Кибернетика и вычислительная техника. — 2018. — № 4 (194). — С. 41-60. — Бібліогр.: 80 назв. — англ. |
| series |
Кибернетика и вычислительная техника |
| work_keys_str_mv |
AT stepashkovs formationanddevelopmentofselforganizingintelligenttechnologiesofinductivemodeling AT stepashkovs formuvannâirozvitoksamoorganízovnihíntelektualʹnihtehnologíiínduktivnogomodelûvannâ AT stepashkovs formirovanieirazvitiesamoorganizuûŝihsâintellektualʹnyhtehnologiiinduktivnogomodelirovaniâ |
| first_indexed |
2025-11-30T09:14:33Z |
| last_indexed |
2025-11-30T09:14:33Z |
| _version_ |
1850206137465438208 |
| fulltext |
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194)
Èíòåëëåêòóàëüíîå
óïðàâëåíèå è ñèñòåìû
DOI: https:// 10.15407/kvt194.04.041
UDC 004.942: 681.513.8
STEPASHKO V.S., DSc (Engineering), Professor,
Head of Dept. for Information Technologies of Inductive Modeling
e-mail: stepashko@irtc.org.ua
International Research and Training Center for Information Technologies
and Systems of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine,
Acad. Glushkov av., 40, Kyiv, 03187, Ukraine
FORMATION AND DEVELOPMENT OF SELF-ORGANIZING
INTELLIGENT TECHNOLOGIES OF INDUCTIVE MODELING
Introduction. Effective solution of control and decision-making tasks in complex systems
should use the results of mathematical modeling. To construct adequate predictive models,
many modern methods and tools are available which may be generally based on two princi-
pal approaches: theory-driven (deductive) and data-driven (inductive) ones. The data-driven
methods are basic for solving typical tacks of data mining; they implement an inductive proc-
ess of transition from particular data to models generalizing the data. Among all such meth-
ods, very notable are those being developed within the area of GMDH-based inductive mod-
eling founded several decades ago by academician O.H. Ivakhnenko.
The purpose of this paper is analysing the background of the GMDH invention by
Ivakhnenko and the evolution of model self-organization ideas, methods and tools during the half-
century historical period of successful development of the inductive modeling methodology.
Results. Professor Ivakhnenko acquired broad knowledge in the areas of automatic con-
trol, engineering cybernetics and emerging neuroscience initiated by the idea of perceptron.
These were those prerequisites which helped Ivakhnenko to synthesize his original self-
organizing approach to solving tasks of constructing models of objects and processes on the
basis of experimental data. The paper tracks evolution of scientific ideas and views of
Ivakhnenko and main achievements in development of GMDH during the period 1968-1997.
Contributions of researchers from different countries to the GMDH modification and appli-
cation are characterized. Results of further developments of inductive modeling methods and
tools in the ITIM department are presented and the most promising prospects of investiga-
tions in this field are indicated.
Conclusions. Main prerequisites facilitating the creation of the GMDH by O.H.
Ivakhnenko were analysed, basic fundamental, technological and applied achievements of the
half-century development of inductive modeling both in Ukraine and abroad were character-
ized, as well as the most prospective ways of further research were formulated.
© STEPASHKO V.S., 2018
41
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 42
Keywords: mathematical modeling, data-driven modeling, model self-organization, GMDH,
inductive modeling, noise-immune modeling, information technology, case study.
INTRODUCTION
Effective solution of control and decision-making tasks in complex systems
should use the results of mathematical modeling. To construct adequate predic-
tive models, many modern methods and tools may be used which generally can
be based on two principal approaches: theory-driven or deductive and data-
driven or inductive ones. The data-driven methods are basic for solving tacks of
data mining; they implement an inductive process of transition from particular
data to models generalizing the data.
Among all data-driven methods, very notable are those being developed in
the area of inductive modeling based on the Group Method of Data Handling
(GMDH) [1-4]. For the scientific school of inductive modeling, founded several
decades ago by Professor Oleksiy Hryhorovych Ivakhnenko, the year 2018 is
reach for a few remarkable jubilees:
− 105 years of Ivakhnenko’s birthday (1913),
− 80 years of his first published article (1938),
− 75 years ago he defended the thesis for the title Candidate of Engineer-
ing Science (1943),
− 50 years of his the very first article on the worldwide-known GMDH (1968),
− 15 years ago he was elected Academician of NAS of Ukraine (2003),
− 20 years ago, the Department for Information Technologies of Inductive
Modeling (ITIM) was established at the IRTC ITS of the NAS and MES of Ukraine.
Taking into account the central item in this list, namely the half-century ju-
bilee of the GMDH origination, which is the main methodological base of Induc-
tive Modeling, it is reasonable to look through the historical period of formation
and development of this scientific direction. Accordingly, this article is intended
to analyze the following aspects:
− attempt to understand which sum of acquired knowledge helps Ivakh-
nenko to approach in 1968 to the creation of GMDH as a self-organizing method
of modeling from experimental data;
− examination of main achievements in development of this induction-
based method during long period from 1968 to 1997;
− appraisal the contribution of researchers from different countries into
evolution of the method;
− explanation of main developments of the ITIM Department in further
progress of the theory, tools and applications of the GMDH-based inductive
modeling during last 20 years, presenting the modern stage of functioning the
inductive modeling school;
− foresight of forthcoming prospects in development of the subject-area of
inductive modeling.
These aspects are comprehensively reflected below in the article and they
differ its matter from the known surveys [1–4] dealing with analysis of various
periods of GMDH and inductive modeling development.
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 43
EVOLUTION OF SCIENTIFIC IDEAS AND VIEWS OF O.H. IVAKHNENKO PRIOR TO THE “GMDH ERA”
Scientific and administrative career. After defending in 1954 the thesis for the
degree Doctor of Engineering Sciences in Kiev Polytechnic Institute and receiv-
ing in 1956 the academic title Professor, O.H. Ivakhnenko has became Head of
laboratory "Automatic control of manufacturing processes" at the Institute of
Electrical Engineering of the Academy of Sciences of the Ukrainian SSR
(1956–1962) reorganized later into the Department of Engineering Cybernetics
at the same institute (1962–1963). In 1961, Ivakhnenko was elected the Corre-
sponding member of the AS of UkrSSR.
In 1963, he with all members of his team has moved to the Institute of
Cybernetics of the Academy of Sciences being invited by the Academician
V.M. Glushkov, and worked there long time as Head of the Department of
Combined Control Systems (1963–1989). After retirement of Ivakhnenko in
1989 from the leading position, the Department was reorganized into the
Laboratory of modeling after experimental data headed by his disciple PhD
G.I. Krotov. In 1995, all the laboratories in the NASU were closed up, and this
team worked in another department as autonomous scientific group.
Based on this group, the Department for Information Technologies of In-
ductive Modeling (ITIM) was established in 1998 at the IRTC ITS of the NAS
and MES of Ukraine headed by his disciple and follower DSc. V.S. Stepashko,
and Professor Ivakhnenko worked there as a Directorate Adviser. In 2003, in the
year of his 90th anniversary, O.H. Ivakhnenko was elected Academician of the
NAS of Ukraine for outstanding achievements in the development of computer
science and artificial intelligence in Ukraine. He fruitfully worked at the depart-
ment up to 2007 when he passed away.
Main stages of the scientific activity. Oleksiy Hryhorovych was an out-
standing scientist in the fields of automatic control, cybernetics and informatics
whose research results in various areas had received worldwide recognition. The
fact is that Ivakhnenko always proves the validity of his views both theoretically
and also by the actual results, namely devices, operating systems and specific
applied tasks solved and it was impossible to not acknowledge such evidences of
rightness of his viewpoint.
Electroautomatics. Prior to the period of the 1960s, the main achievements
of O.H. Ivakhnenko concern areas of electroautomatics and the invariance the-
ory: he has developed a new principle of automatic speed control of alternating
current motors using magnetic amplifiers, asynchronous motors by the differ-
ence of voltage and current etc. [5].
Combined systems of automatic control. O.H. Ivakhnenko developed the theo-
retical basis of invariant automatic control systems that operate by the principle of
compensation of the measured disturbances. He is the author of a new for automatic
control theory principle of combined control based on the negative feedback with
respect to the controlled variable and the positive (compounding) feedforward by con-
trolled disturbances. This principle allowed for the first time to create real invariant
systems of automatic control for asynchronous motors speed [6].
Control systems with self-tuning, self-learning, pattern recognition and
forecasting. O.H. Ivakhnenko proposed to use a variety of devices and methods
of self-tuning in combined control systems. He published in 1959 the very first
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 44
in the former USSR monograph on engineering cybernetics [7] where the cyber-
netic control systems are considered which differ from the conventional automa-
tion systems by the presence of automatic self-modification, self-adaptation and
self-tuning of characteristics when external and internal influences change. This
book brought him international recognition and was reprinted in English and
some other European and Asian languages. He was invited to give lectures in
Europe and the USA; the "father of cybernetics" Norbert Wiener visited Kyiv in
1962 at his invitation.
In 1960th, working at the Institute of Cybernetics, O.H. Ivakhnenko started
to use principles of self-learning, pattern recognition and forecasting when de-
signing control devices. Under his leadership, the recognition system "Alpha"
was constructed which demonstrated the self-learning process at first time in the
world. His monograph "Cybernetic forecasting devices" [8] was re-released in
English, Polish, German and Chinese.
The sum of this broad and diverse knowledge acquired by him personally and
supplemented by deep understanding the possibilities of the perceptron introduced by
Frank Rosenblatt [9] were those prerequisites which help Ivakhnenko to synthesize his
original self-organizing approach to solving tasks of constructing models of objects
and processes on the basis of experimental data. This development gave start to new,
long and very fruitful period of his scientific creativity.
MAIN ACHIEVEMENTS IN DEVELOPMENT OF GMDH DURING THE PERIOD 1968-1997
Heuristic self-organization of models. The very first article [10] on the group
method of data handling (GMDH) published in 1968 in Ukrainian journal “Av-
tomatyka”, reprinted in the USA [11] and followed by publications [12, 13] abroad,
signified the beginning of the most fruitful stage in the creative scientific work of
O.H. Ivakhnenko. It is worth to no note that in Ukrainian language this method is
called «Метод групового урахування аргументів» (МГУА), and direct translation
would give an accordant and correct English equivalent “Method of Group Using of
Arguments” (MGUA). But when translating the article [10] in the USA journal “So-
viet automatic control”, the reprinted version of “Avtomatyka”, a specific name
“Group Method of Data Handling” was introduced and the abbreviation GMDH is
now world-wide known as the method, explicitly associated with its creator.
With these publications, a new scientific area has emerged that was named at
first by Ivakhnenko as “heuristic self-organization of models” [14]. This name had
several modifications later: “self-organization of models from experimental data”,
further “inductive self-organization of models” [15], then “inductive learning algo-
rithms” [16], and finally, from the 1998, simply “inductive modeling” (see below).
Active development of this area was determined by brilliant scientific intuition of
the author and his original scientific hypotheses. Since 1968, this new scientific field
is steadily and successfully developing being recognized, adopted and used by re-
searchers around the world.
Generally speaking, the task of inductive modeling consists in an automated
data-based construction of a mathematical model, approaching an unknown regular-
ity of functioning the simulated object or process. GMDH is an original self-
organizing inductive method of modeling from experimental data, or structure and
parametric identification of forecasting models, first of all of polynomial type, under
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 45
uncertainty conditions. It is notable for application of the following main principles for
solving optimization tasks of parameter estimation and structure determination:
− automatic generation of groups of model variants with various complexity;
− inductive way of model construction with evolution from simple to more
complex model structures;
− evaluation of the model quality by external criteria based on the division of a
dataset into at least two parts: the first one is used for parameter estimation and the
second for criteria calculation;
− non-final decisions during the evolutionary modeling process: at any stage or
iteration, not unique the best model is selected but a subset of best ones;
− successive selection of models of optimum complexity with respect to mini-
mal values of the external criteria.
GMDH is intended for automatic building a mathematical model that repro-
duces the unknown regularity of operation of the investigated object information
on which is implicitly contained in the available data sample (table). After the
modern terminology this is an original means of data mining.
The method effectiveness was repeatedly confirmed by the solution of vari-
ous of practical problems of modeling complex objects and processes in areas of
ecology, economy, engineering, technology, hydrometeorology, etc. [14–16]. In
parallel, researches were conducted on the development and application of self-
organization algorithms in pattern recognition tasks [17].
General characteristic of the model self-organizing task. As a rule, GMDH
deals with building models in the class of functions linear in parameters. Let us
be given a data sample ][ yXW = of n observations over m input variables
mjnixX ij ,1,,1],[ === , and one output T
nyyy ),...,( 1= . Suppose that we may
build a set Φ of various models of the form
)( ff
ˆ,Xfŷ θ= , Φ∈f , (1)
using the data set W, where the parameter estimation fθ̂ for each f ∈Φ is the
solution of a task of continuous minimization of an error criterion )(⋅QR :
)( f
R
f ,y,XQRminargˆ
fs
f
θ=θ
∈θ
(2)
where fs is the complexity of the model (1), or the number of unknown pa-
rameters of the model.
Consequently, the problem of the best model constructing from the given
experimental data consists then generally in finding the minimum of a given
criterion )(⋅CR on the discrete set Φ of models of different structures:
)( f,y,XCRminarg*f
f Φ∈
= (3)
The two criteria )(⋅QR and )(⋅CR are called in the GMDH theory as internal
and external ones respectively. They should be different, )()( ⋅≠⋅ CRQR , because
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 46
in another case the task (3) has trivial solution: *f corresponds to the model of
maximum complexity maxs . It is evident especially in the case of linear func-
tions, when the task (2) is solved using the Least squares method (LSM) for
estimation of parameters, and the LSM is a monotonically decreasing function of
the number of parameters.
To ensure the condition )()( ⋅≠⋅ CRQR , in GMDH the data division into at
least two parts is used:
⎥
⎦
⎤
⎢
⎣
⎡
=⎥
⎦
⎤
⎢
⎣
⎡
=
BB
AA
yX
yX
B
A
W , (4)
then the internal criterion has the form ),,( fAA yXQR θ when the external one
)( AfBB
ˆ,y,XCR θ : they are evidently different, and the task (3) is not trivial.
In general, the process of solving the problems of structural and parametric
identification (1–3) includes the following main stages:
1) setting a data sample and a priori information;
2) the choice or assignment of a class of basic functions and the respective
transformation of data;
3) generation of different model structures in this class;
4) estimation of parameters of generated structures by an internal criterion
)( fQR θ and formation of the set Φ ;
5) minimization of a given external criterion CR(f) and selection of the op-
timal model f*;
6) checking the adequacy of the model obtained;
7) completion of the process and/or the use of the model.
This means that each modeling method including GMDH can be described with
the help of four main components: a) class of basic functions, b) structure generator,
c) method for estimating parameters, and d) criterion for model selection. As far as
GMDH is dealing mainly with model classes linear in parameters, the use of LSM-
estimator is the best option. Hence, there is a need to analyze the formulas for exter-
nal criteria and the types of model structure generators in GMDH algorithms.
One can use two main external criteria for model selection based on the
sample division (4). The first one is called the regularity criterion which for
linear models equals to the model error on the data subset B when parameters
were estimated on the subset A:
2
)( ABBABBB
ˆXy,̂y,XAR θ−=θ . (5)
And another one is the unbiasedness criterion, evaluating the difference be-
tween outputs of two models of the same structure with parameters estimated on
the parts A and B:
2
BWAW
ˆXˆXCB θ−θ= . (6)
Main types of GMDH algorithms. Any of the GMDH algorithms realizes
some kind of a self-organizing process of sequential generation of more and
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 47
more complex models in an inductive manner: any next such complication takes
into account results of the previous stage of model building and may by based on
both adding some new members to the previous model structures and combining
them using some superposition function called “partial description”. This self-
organizing process of model complication stops when the minimum value of a
given external criterion is reached.
Consequently, all the GMDH algorithms are generally divided into two big sepa-
rate groups: the sorting-out [18] and iterational [19] ones that differ from each other
by the methods of model structures generation and the procedures of search of mini-
mum of a selection criterion based on a data sample division into two or more parts.
Sorting-out GMDH algorithms. The first type algorithms generate models of
various structures in a given set of basic functions like members of polynomial
of a preset power or lag terms of an autoregression model. They all are based on
the exhaustive or directed search of the model of optimal complexity.
The most known combinatorial algorithm is called COMBI [20] generating
all possible model structures:
m
vvv ,...,v,ˆXy 21=θ= , (7)
where the decimal number ν is assigned a binary structure vector dv, in which
units indicate the presence of arguments, when zeros their absence. Here, the
number of models being compared and forming the set Φ equals to 2m and the
exhaustive search is effective only up to approximately m = 30 because of expo-
nential growth of calculations.
The purpose of the directed search methods is to find the global minimum
of the selection criterion, i.e. the result of an exhaustive search, by significantly
smaller calculations. They are runnable with hundreds of arguments, e.g. the
multistage algorithm MULTI [21] which uses a special step-by-step evolving
procedure of the type
11 11)( −− ==θ= ss
j
s
i
s
l
s F,l,i,m,s,ˆx|Xŷ , (8)
where s is the stage number (and the structure complexity); Fs — number of the
best structures (freedom of choice) at the stage s; j is the index of an argument
vector being added to the matrix i
sX 1− . In this case, a subset Φ⊆Φs is analyzed,
with a high probability containing the result of a complete search. This algorithm
has the polynomial complexity of order m3.
Iterative GMDH algorithms. The second type algorithms, being classical,
are based on the nature inspired idea of biological selection with pairwise ac-
count of features. According to the principle of operation, the iterative GMDH
algorithms are similar to methods of optimization using successive approaching,
but the principle of indecisive solutions (freedom of choice F) is here substan-
tially used. Depending on the method of successive approaching, they may be
divided into two main groups: multilayered and relaxational ones.
The conventional is the so-called multilayered algorithm [11–13] which
starting from late 1990th is commonly called in the current literature as multilay-
ered iterative algorithm MIA GMDH. It is the algorithm that was initially cre-
ated in 1968 by Ivakhnenko and named multilayered algorithm MGUA/GMDH.
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 48
It has the perceptron-like architecture and constructs hierarchical network-type
models through an iterational procedure of step-by-step complicating model
structures: a partial model of a current layer/iteration is a polynomial of any pair
of models of the previous layer.
The MIA GMDH was constructed by the analogy with the biological selec-
tion of living organisms: the complication of models from r-th layer to (r+1)-th
occurs due to the "crossing" of all possible pairs from F best models of the pre-
vious layer. Typically, the MIA partial description is of the form:
21 ,1;,1,,...;1,0),,( F
r
j
r
il
r
l ClFjiryyfy ====+ , (9)
where second order polynomial is usually used, but bilinear or even linear one
may be applied as well. The iterative process of model complication stops after
the criterion CR starts to increase.
During 1970th, there were developed a lot of various modifications of the classi-
cal algorithm; the most original among them are: multilayered algorithm with or-
thogonalization of arguments [22], linear CML [23] and nonlinear GN [24] algo-
rithms. They were generally dissimilar but had common features separating them
from the classical type algorithm: they have the only two-member partial descrip-
tions and the use of initial arguments on each layer of selection. A GMDH proce-
dure of such type (based on adding initial arguments to the intermediate ones) is
called now relaxational iterative algorithm RIA GMDH.
In GMDH algorithms of the RIA type, models are complicated on each
layer by "crossing" the best models of the previous layer with the initial argu-
ments, and a partial description of the following modification:
mkFixyfy k
r
il
r
l ,1;,1),,(1 ===+ , (10)
which may be used with quadratic, bilinear or linear polynomial.
Such kind of descriptions was introduced in [22–24] in the form
mkFixyy k
r
i
r
l ,1;,1),(1 ==ϕ+=+ to avoid some known drawbacks of the MIA
GMDH: e.g., they help to exclude the possibility of losing relevant arguments.
In general, iterative GMDH algorithms are operable at m > 1000, and they
allow constructing models even in degenerate problems when n < m. Historically
the very name of GMDH is associated primarily with them.
Expressions (7–10) represent in a simplified form the four main types of
structure generators in GMDH algorithms. All of them build models that are
linear in parameters but generally non-linear in inputs. Typical GMDH applica-
tions are modeling of nonlinear systems, forecasting complex processes, func-
tion approximation, recognition, classification, clustering and others.
Control with optimization of forecast. In 1970th, O.H. Ivakhnenko had de-
veloped a new method of control with the forecast optimization [25] (optimiza-
tion on a sliding finite interval) which was fundamentally necessary in control
problems of environmental systems as the tasks with predicted disturbances. A
control system with such an optimizer is a combined one: in the implemented
control action the requirements are taken into account of both stabilization of the
output value and advanced compensation of the measured disturbance that at-
taches intelligent features to the control system. Such an approach called cur-
rently "predictive control" had started to develop only in the late 1980th.
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 49
Noise-immune modeling. In early 1980th O.H. Ivakhnenko had established
an organic analogy between the tasks of model construction from noisy data and
signal passing through a noisy channel [26]. This had made it possible, using
Kotelnikov’s ideas on the noise-immune receiver of signals, to suggest the so
called “ideal criterion” for modeling tasks that enabled to proof theoretically
better forecasting capabilities of “non-physical” models [27] and to build fun-
damentals of the noise-immune modeling theory [18]. The basic result of the
theory: the complexity of optimum model depends on the level of uncertainty —
the higher it is the simpler (more «robust») is to be the optimum model.
Two-level predictive models. For modeling of dynamics of cyclic processes,
in particular economic and ecological, O.H. Ivakhnenko suggested a method for
construction of multilevel forecast systems [25]. The theoretical base of the two-
level modeling was presented in [18], and an appropriate algorithm for construc-
tion of two-level difference models for cyclicities of the “cycle-time” type was
described in [28]: the variables of upper (cycle-averaged) and lower (time-
averaged) levels in their balance relationship are thus considered.
Non-parametric forecasting. O.H. Ivakhnenko developed original non-
parametric approaches to forecasting complex processes: correlation predictive
models; a reverse transformation of the transition probabilities matrix into fore-
cast [29]; a method of the group analogues complexing [30].
Ideas of systems analysis and clustering. An algorithm of objective system
analysis (OSA) is intended for determination of inputs and outputs variables
among all measured ones and building the system of difference equations of an
object dynamics [31]. An algorithm of the objective computer clusterization
(OCC) solves the tasks of clustering with the use of self-organization of the
quantity and composition of clusters [32].
Modeling with other basic functions. O.H. Ivakhnenko promoted evolving
new algorithms of model construction in such model classes: harmonic models
of oscillatory processes in the form of sum of harmonics with aliquant frequen-
cies [33]; nonlinear additive-multiplicative models with automatic determination
of non-integer degrees of input variables [34]; etc.
Analysis of selection criteria, optimal complexity of models. In the GMDH
theory, the scope is first of all on analysis of selective properties of external cri-
teria which is carried out within the framework of the noise-immune modeling
theory. The initial results [26, 27, 18] were supplemented in [35, 36] by studies
of so-called J-optimality (J-criterion is another name for the “ideal” one [27]).
Theoretical results regarding the problem of convergence analysis of itera-
tive GMDH algorithms one can find in [37, 23, 19]. Another important aspect of
the GMDH theory was concerned the analysis of asymptotic properties of exter-
nal criteria [38, 39]. Main results of O.H. Ivakhnenko and his colleagues during
1980th are reflected both in the domestic books [18, 19] and abroad [16].
GMDH as the polynomial neural network. In 1990th, the typical MIA
GMDH procedure was interpreted as an original architecture of a neural net
called polynomial neural network (PNN). The point is that the originality of
GMDH as a neuronet consists in self-organization of both its structure and pa-
rameters. To the explicit advantages of GMDH we can refer automatic forma-
tion of a network structure, simplicity and speed of parameters adjustment as
well as the possibility to "fold" the adjusted network directly into an explicit
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 50
mathematical expression ready to be used for solving tasks of simulation, predic-
tion, control and decision making.
To illustrate the originality of the MIA GMDH as an automatically con-
structed PNN architecture, Fig. 1 represents a very simple example of the net-
work state (for the case of 4 input arguments): a) after stopping the forward it-
erative process of the best model *f building, and b) the resulting trained
GMDH network after backward tuning. As the result of the net self-organization,
the argument x2 appears to be redundant. As it is evident, the tuned net may be
easily reduced to direct formula ),,(* 431 xxxff = .
Neuronet with active neurons. A typical "GMDH neuron" in the form of
quadratic polynomial of two arguments can be called as "passive" because any
of the neurons have the same fixed structure, i.e. the PNN GMDH is homogene-
ous net. In the 1990th, Ivakhnenko proposed a new type of GMDH network with
active neurons [40] or a heterogeneous network in which any of the neurons is
in turn also a GMDH algorithm, due to that the structure of such neuron is opti-
mized. As a result, all neurons can get different structures increasing the flexibil-
ity of configuring the network to a specific task. Networks of such type are also
called as "twice multilayered" ones [41].
a)
b)
Fig. 1. An example of self-organization of the GMDH net architecture
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 51
DEVELOPMENT OF IVAKHNENKO’S IDEAS BY RESEARCHERS OF THE WORLD
Scientific results of O.H. Ivakhnenko are widely known in Ukraine and abroad:
more than 40 books and 500 scientific papers were published by him, many of
which, in the first place books, was republished in English, German, Polish,
Bulgarian, Romanian and Chinese (11 monographs published abroad). Graphic
evidence of the global recognition of merits of this method and its author was
the publication in 1984 in the United States of the monograph of American and
Japanese researchers on theoretical and applied aspects of GMDH [42] dedicated
to the 70th anniversary of the GMDH author.
Hybrid GMDH-type algorithms and neural networks. New and efficient archi-
tectures of neuronets are recently intensively developed on the basis of hybridization
of GMDH procedures and various approaches of computational intelligence and
nature-inspired solutions: particle swarm optimization [43], RBF [44], genetic selec-
tion and cloning [45], immune systems [46] etc. Several variants of hybridization are
implemented in the GMDH-based architecture called GAME [47].
Fuzzy and interval approaches in inductive modeling. For the real tasks with
fuzzy variables there have developed corresponding algorithms: Multi-layer hybrid
fuzzy PNN [48]; Fuzzy GMDH [49] based on the classical structure MIA GMDH;
GMDH-like cascade wavelet-neuro-fuzzy network [50]. For another case of initial
assumption on interval-given input data, a method [51] has elaborated.
Algorithms based on paralleling operations. There are several effective re-
alizations of both iterative [52] and combinatorial [53] algorithms with imple-
mented parallel computations.
Ensembling-based prediction. The core idea in the GAME method [54] is a
kind of non-parametric procedures for predicting processes using not a separate
model but a weighted average of assemblage of several predictions obtained by
various GMDH-type algorithms.
Automated data preprocessing. An advanced idea is suggested in [55] to
build some kind of automated means for data preprocessing enabling to enhance
the accuracy of classification.
Modeling tasks with multiple outputs. New analytical and applied results
were obtained for the task of inductive modeling of multidimensional systems
with many outputs [56].
Inductive algorithms for classification, recognition, clusterization. Essential part
of real-world problems requires application of special methods of pattern recognition
in wide sense. Among current developments in the inductive modeling field there are
many methods and tools dealing with tasks of such type, for instance [47, 57].
Software tools based on GMDH. There are several examples of computer
tools for modeling complex systems completely based on GMDH and/or
GMDH-type algorithms. The software complex KnowledgeMiner of commercial
destination and its recent highly advanced realization [58] contains iterative and
several non-parametric GMDH algorithms for fully automatic building and ana-
lyzing forecasting models. Another variant of commercial software is
GMDHshell [59] based on COMBI and MIA GMDH algorithms. The technol-
ogy FAKE GAME [47] is intended for construction, investigation and applica-
tion of inductive evolutionary algorithms of different architectures.
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 52
Real-world applications of inductive modeling. During the whole period of
GMDH evolution this method demonstrated good performance when solving
modeling problems for complex processes and systems of different nature in
environment, economy, finance, hydrology, technology, robotics, sociology,
biology, medicine, and others. Many examples of such modeling results can be
found on the ITIM department site [60] in papers presented at conferences and
workshops on inductive modeling as well as in the annual issue of collected
papers “Inductive modeling of complex systems”.
FURTHER DEVELOPMENTS OF INDUCTIVE MODELING METHODS
AND TOOLS IN THE ITIM DEPARTMENT
In 1998, precisely 20 years ago, when the Department for information technolo-
gies of inductive modeling (ITIM) was established at the IRTC ITS of NAS of
Ukraine, the scientific direction created by O.H. Ivakhnenko obtained its short
name «Inductive Modeling» being now generally accepted for scientific forums,
articles and books in Ukraine and abroad. This department is the only one in the
world which is professionally dealing with the theoretical, technological and
applied investigations in this field. Its activity presents the modern stage of func-
tioning the Inductive modeling school and provides the coordination of studies
in this area by annually holding International Conferences and Workshops on IM
as well as Summer schools for young researchers devoted to issues of the
GMDH theory, algorithms and applications. These events take place both in
Ukraine and abroad and involve researchers of many countries.
The research strategy of the department covers the full life cycle of a scien-
tific research: the methodology of modeling from data samples; the theory of
inductive construction of models of optimal complexity; algorithmization of
high-performance modeling tools; intellectualization of technologies of con-
structing models; computer experiments to evaluate the effectiveness of devel-
oped technologies; solving real problems of modeling and forecasting; applied
implementation of the developed software tools in monitoring, control and deci-
sion support systems.
The term inductive modeling is defined as a self-organizing process of evolu-
tional transition from initial data to mathematical models reflecting some functioning
patterns of the modeled objects and systems implicitly contained in the available ex-
perimental, observational or statistical information under the uncertainty conditions
[5]. Instead of the traditional deductive way of solution of modeling tasks «from gen-
eral theory — to particular model», inductive approach is based on the intensive use of
computers as the way «from particular data – to general model».
Main scientific results of the ITIM department may be characterized as follows.
Theory of inductive modeling. Detailed outcomes concerning conditions of
constructing models of optimal complexity were presented in [61] using so called
method of critical variances. This main analytical instrument of the inductive
modeling theory has made it possible to prove that GMDH is the method of con-
struction of models with minimum variance of the forecasting error. GMDH proc-
ess enables an automatic adaptation of the optimal model to the level of informa-
tivity of a noisy data sample. This makes it possible to classify GMDH as an effec-
tive method of data mining and computational intelligence, since it is aimed at
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 53
automatic search and description of regularities with the choice of structure and
parameters of linear, nonlinear, difference and other models based on data samples
under condition of uncertainty and incompleteness of a priory information.
Enhanced and optimized GMDH algorithms. An original two-criterion
method of redefinition (additional determination) the optimal model choice was
elaborated in [62] which enhances the reliability of model building from data.
High-performance sorting-out GMDH algorithms were designed and im-
plemented based on recurrent calculations [63], paralleling operations [64],
sequential selection of informative variables [65], and genetic procedures of
model construction [66]. They all allow enhancing the dimensionality of the
real-world problems being solved.
New hybrid architectures of iterative GMDH algorithms were constructed
as a generalization of algorithmic structures of multilayered, relaxational and
combinatorial types, based on which the generalized iterative algorithm GIA
GMDH [67] was developed as a neural network with active neurons in the form
of the COMBI algorithm for automatic adjustment of a neuron complexity.
The generalized relaxational iterative algorithm GRIA GMDH was devel-
oped based on the use of high-speed recurrent computations of parameters and
external criteria during the multilayer procedure, which allows solving inductive
modeling problems from high-dimension data [68].
Theoretical foundations of intelligent modeling of complex processes, e.g. so-
cioeconomic ones, under conditions of incompleteness and uncertainty of a priori
information are developed based on the use of knowledge bases, means of induc-
tive data analysis and intelligent user interface [69].
An original approach to substantial enhancing the effectiveness of sorting-
out GMDH algorithms is based on combined implementation of recurrent and
parallel calculations [70].
Developed instrumental tools. Generally all GMDH-based means allow
solving typical tasks of data mining, knowledge discovery and business intelli-
gence. The following tools have been developed:
− complex of software tools [71] for designing, researching and applying
modeling methods, conducting experiments on testing modeling methods and
their components (model classes, generators of model structures, methods for
estimating parameters and models selection criteria).
− software package ASTRID-GIA [72] for inductive modeling of complex
systems based on various iterative GMDH algorithms makes it possible to use
the generalized algorithm GIA GMDH and all its special cases [69] in online
access mode both over the Internet and in the local network.
− computer system of automated structure and parameter identification ASPIS
[73] for building predictive models on the basis of the high-speed generalized relaxa-
tional iteration algorithm GRIA GMDH [68] for solving big data problems.
− software tool for modeling and predicting complex interrelated proc-
esses on the basis of recurrent-and-parallel GMDH algorithms in the class of
dynamic models of vector autoregression [74];
− management decisions informational support system MDISS for solving
problems of estimation, analysis and forecasting of the state of complex systems
of interrelated socio-economic processes with the purpose of making reasonable
managerial decisions [75].
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 54
Many of the mentioned above results of ITIM department are reflected in
the three monographs dealing with: elaboration, computer-based research and
practical application of high-performance sorting-out GMDH algorithms [76];
design, theoretical study, numerical testing and application of the GMDH algo-
rithms of the relaxational type [77]; development and investigation of the gener-
alized iterative algorithm GIA GMDH, construction the interactive software
package ASTRID-GIA [78].
Basic application results. Various tasks of modeling of complex systems
and processes of different nature have been solved, in particular:
− dynamics of interdependent indicators of the energy and investment areas
in the class of vector autoregression models for a short-term forecast [74, 75];
− dependence of the sputtering (disruption) coefficient of a spacecraft surface
under action of ionized gas jets on the physical properties of the surface coating [76];
− predicting the results of testing blood samples with medicines to deter-
mine the most effective for a patient; construction of classifiers for the differen-
tial diagnostics of blood diseases [77].
− quantitative assessment of the impact of sea water pollution with bitu-
moid substances on the total number of species of benthic organisms in the Se-
vastopol bays [78];
− modeling of the quantity of microorganisms in the soil depending on
weather factors and the dose of contamination with heavy metals [79].
PROSPECTS OF RESEARCH DEVELOPMENT IN THE FIELD OF INDUCTIVE MODELING
The most promising directions of the research development may be indicated as
follows:
− theory of intellectualization of inductive modeling tools using paradigms
of neural networks with active neurons and hybrid architectures of GMDH with
evolutionary and multiagent methods;
− theory and high-performance structures of the sorting-out GMDH algo-
rithms on the basis of recurrent-and-parallel and evolutionary calculations;
− methodology, theory and intelligent algorithms for inductive solving
tasks of classification and clustering for detecting regularities and automatic
analysis of big data;
− intelligent technologies for informational support of making decisions in
systems of various nature on the basis of instrumental means of interactive syn-
thesis of inductive modeling algorithms;
− optimal preprocessing procedures to improve the modeling results;
− non-parametric modeling and forecasting based on waited averaging of
model ensembles;
− knowledge-based intelligent user interface with strong support and con-
trol of user’s activity;
− application of the developed means of intelligent modeling in applied
problems of analysis and forecast of socioeconomic, ecological and technologi-
cal processes;
− theory and application of tools for informational support of the decision-
making based on inductive modeling methods and algorithms;
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 55
− knowledge discovery based on inductive procedures of automated con-
tent analysis of messages on Internet, in social networks and textual sources;
It should be noted that the research community dealing with the problem of
Deep Learning [80] has recognized that the GMDH created in 1968 by Ivakhnenko
is the very first example of a deep learning network with self-organization of its
structure, parameters and depth. This fact opens new and very promising prospects
for further research developments in the field of inductive modeling.
CONCLUSIONS
An enormous amount of articles has been published during the 50-years period
in the world on the subject of GMDH-based inductive modeling. In this article,
structured information was presented on historical aspects of the formation and
development of inductive modeling as original scientific direction in Ukraine
and abroad, including the ITIM department. Main prerequisites facilitating the
creation of the GMDH by O.H. Ivakhnenko were analyzed, basic fundamental,
technological and applied achievements of the half-century development of in-
ductive modeling both in Ukraine and abroad were characterized, as well as the
most prospective ways of further research were formulated.
GMDH as the main instrument of inductive modeling is an original and ef-
fective means of solving a wide range of problems of artificial intelligence. The
structure of GMDH is interpreted as a neural network whose originality consists
in the self-organization of both its structure and parameters, and it is defined as a
kind of architecture of deep learning. Inductive modeling theory refers to the
most modern methods of data mining, computation intelligence and soft comput-
ing. GMDH-based self-organizing inductive modeling tools represent an original
and efficient facility for solving a wide range of artificial intelligence prob-
lems including identification and forecast, pattern recognition and clusteriza-
tion, data mining and search for regularities.
The main objective of scientific research of the ITIM department is developing
theoretical and applied bases of designing intelligent tools and technologies for induc-
tive modeling of dynamics of complex interrelated processes based on statistical data
for informational support of management decisions in digital economy systems. This
objective corresponds to the dominating world trends in the development of technolo-
gies for high performance computing, deep learning and big data analysis.
REFERENCES
1. Ivakhnenko A.G., Müller J.-A. Recent Developments of Self-Organizing Modeling in
Prediction and Analysis of Stock Market. Microelectronics Reliability. 1997. No. 37.
P. 1053–1072.
2. Anastasakis L., Mort N. The Development of Self-Organization Techniques in Model-
ling: A Review of the Group Method of Data Handling (GMDH). ACSE Research Re-
port 813. The University of Sheffield, 2001. 39 p.
3. Snorek M., Kordik P. Inductive Modelling World Wide the State of the Art. Proc. of 2nd Int.
Workshop on Inductive Modelling (Prague, 19—23rd of Sept., 2007) Prague, 2007. P. 302–304.
4. Stepashko V. Developments and Prospects of GMDH-Based Inductive Modeling. In:
Advances in Intelligent Systems and Computing II: Selected Papers from the Intern.
Conf. on Computer Science and Information Technologies, CSIT 2017, Lviv, Ukraine /
N. Shakhovska, V. Stepashko, Editors. AISC book series, Vol. 689. Cham: Springer,
2018, pp. 474–491.
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 56
5. Ivakhnenko A.G. Electroautomatics. Kiev: Gostekhizdat UkrSSR, 1957. 452 p. (In Russian)
6. Ivakhnenko A.G., Petina N.V. Voltage stabilizers with combined control. Kiev: AS
UkrSSR publisher, 1958. 247 p. (In Russian)
7. Ivakhnenko A.G. Engineering cybernetics. Kiev: Gostekhizdat UkrSSR, 1959. 432 p. (In
Russian)
8. Ivakhnenko A.G., Lapa V.G. Cybernetic predicting devices. Kiev: Naukova dumka,
1965. 213 p. (In Russian)
9. Rosenblatt F. Principles of Neurodynamic: Perceptrons and the Theory of Brain Mecha-
nisms. Washington: Spartan Books, 1962. 616 p.
10. Ivakhnenko A.G. Method of Group Using of Arguments as a Rival of Stochastic Ap-
proximation Method. Avtomatyka. 1968. № 3. P. 58–72. (In Ukrainian)
11. Ivakhnenko A.G. Group Method of Data Handling as a Rival of Stochastic Approxima-
tion Method. Soviet Automatic Control. 1968. Nо. 3. P. 43–55.
12. Ivakhnenko A.G. Heuristic Self-Organization in Problems of Automatic Control. Auto-
matica (IFAC). 1970. No. 6. P. 207–219.
13. Ivakhnenko A.G. Polynomial theory of complex systems. IEEE Trans. Sys., Man and
Cyb. 1971. 1, No 4. P. 364–378.
14. Ivakhnenko A.G. Heuristic self-organization systems in engineering cybernetics. Kiev:
Tekhnika, 1971. 392 p. (In Russian)
15. Ivakhnenko A.G. Inductive method of self-organization of complex systems. Kiev:
Naukova dumka, 1982. 296 p. (In Russian)
16. Madala H.R., Ivakhnenko A.G. Inductive Learning Algorithms for Complex Systems
Modeling. London, Tokyo: CRC Press Inc., 1994. 384 p.
17. Ivakhnenko A.G., Zaichenko Yu.P., Dimitrov V.D. Decision making based on self-
organization. Moscow: Sov. radio, 1976. 280 p. (In Russian)
18. Ivakhnenko A.G., Stepashko V.S. Noise-immunity of modeling. Kiev: Naukova dumka,
1985. 216 p. (In Russian)
19. Ivakhnenko A.G., Yurachkovskiy Yu.P. Modeling of complex systems from experimen-
tal data. Moscow: Radio i svyaz, 1987. 120 p. (In Russian)
20. Stepashko V.S. A Combinatorial Algorithm of the Group Method of Data Handling with
Optimal Model Scanning Scheme. Soviet Automatic Control. 1981. 14(3). P. 24–28.
21. Stepashko V.S. A Finite Selection Procedure for Pruning an Exhaustive Search of Mod-
els. Soviet Automatic Control. 1983. 16(4). P. 88–93.
22. Shelud'ko O.I. GMDH Algorithm with Orthogonalized Complete Description for Syn-
thesis of Models by the Results of a Planned Experiment. Soviet Automatic Control.
1974. 7(5), pp. 24–33.
23. Yurachkovsky Yu.P. Convergence of Multilayer Algorithms of the Group Method of
Data Handling. Soviet Automatic Control. 1981. 14(3). P. 29–34.
24. Yurachkovsky Yu.P. Restoration of Polynomial Dependencies Using Self-Organization.
Soviet Automatic Control. 1981. 14(4). P. 17–22.
25. Ivakhnenko A.G. Long-term forecasting and control of complex systems. Kiev:
Tekhnika, 1975. 311 p. (In Russian)
26. Ivakhnenko A.G., Karpinsky A.M. Computer-Aided Self-Organization of Models in
Terms of the General Communication Theory (Information Theory). Soviet Automatic
Control. 1982. 15(4). P. 7–15.
27. Stepashko V.S. Potential noise stability of modelling using the combinatorial GMDH
algorithm without information regarding the noise. Soviet Automatic Control. 1983.
16(3). P. 15–25.
28. Stepashko V.S., Kostenko Yu.V. A GMDH Algorithm for Two-level Modeling of Mul-
tidimensional Cyclic Processes. Soviet Automatic Control. 1987. 20(4). P. 49–57.
29. Ivakhnenko A.G., Osipenko V.V., Strokova T.I. Prediction of Two-dimensional Physical Fields
Using Inverse Transition Matrix Transformation. Soviet Automatic Control. 1983. 16(4). P. 10–15.
30. Ivakhnenko A.G. Inductive Sorting Method for the Forecasting of Multidimensional
Random Processes and Events with the Help of Analogs Forecast Complexing. Pattern
Recogn. and Image Analysis. 1991. 1(1). P. 99–108.
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 57
31. Ivakhnenko A.G. Kostenko Yu.V. System Analysis and Long-Term Prediction on the Basis of
Model Self-organisation (OSA algorithm). Soviet Automatic Control. 1982. 15(3). P. 11–17.
32. Ivakhnenko A.G. Objective Computer Clasterization Based on Self-Organisation The-
ory. Soviet Automatic Control. 1987. 20(6). P. 1–7.
33. Vysotskiy V.N., Ivakhnenko A.G., Cheberkus V.I. Long Term Prediction of Oscillatory
Processes by Finding a Harmonic Trend of Optimum Complexity by the Balance-of-
Variables Criterion. Soviet Automatic Control. 1975. 8(1). P. 18–24.
34. Ivakhnenko A.G., Krotov G.I. A Multiplicative-Additive Nonlinear GMDH Algorithm with
Optimization of the Power of Factors. Soviet Automatic Control. 1984. 17(3). P. 10–15.
35. Kocherga Yu.L. J-optimal Reduction of Model Structure in the Gauss-Markov Scheme.
Soviet J. of Automation and Information Sciences. 1988. 21(4). P. 34–36.
36. Aksenova T.I., YurachkovskyYu.P. A Characterization at Unbiased Structure and Conditions of
Their J-Optimality. Sov. J. of Automation and Information Sciences. 1988. 21(4). P.36–42.
37. Ivakhnenko A.G., Kovalchuk P.I., Todua M.M., Shelud'ko O.I., Dubrovin O.F. Unique
Construction of Regression Curve Using a Small Number of Points — Part 2. Soviet Au-
tomatic Control. 1973. 6(5). P. 29–41.
38. Stepashko V.S. Asymptotic Properties of External Criteria for Model Selection Soviet
Journal of Automation and Information Sciences. 21, Nо. 6. (1988). P. 84–92.
39. Aksenova T.I. Sufficient conditions and convergence rate using different criteria for model
selection, Systems Analysis Modelling Simulation 1995. vol. 20, no. 1–2. P.69–78.
40. Ivakhnenko A.G., Ivakhnenko G.A., Mueller J.A. Self-Organization of Neuronets with
Active Neurons. Pattern Recognition and Image Analysis. 1994. 4(4). Р. 177–188.
41. Muller J.-A., Lemke F. Self-organizing data mining. An intelligent approach to extract
knowledge from data. Berlin, Dresden: Libri BoD, 1999. 225 p.
42. Self-organizing methods in modeling: GMDH type algorithms / Ed. S.J. Farlow. New
York, Basel: Marcel Decker Inc., 1984. 350 p.
43. Voss M.S., Xin Feng. A new methodology for emergent system identification using particle
swarm optimization (PSO) and the group method of data handling (GMDH). Proc. of the
Genetic and Evolutionary Computation Conference. Morgan Kaufmann Publishers, 2002.
(9–13th of July, 2002, New-York). New-York, 2002. P. 1227–1232.
44. Kondo T., Ueno J. Feedback GMDH-Type Neural Network Self-Selecting Optimum
Neural Network Architecture and Its Application to 3-Dimensional Medical Image Rec-
ognition of the Lungs. Proc. of the II Intern. Workshop on Inductive Modelling IWIM-
2007 (Prague, 19-23rd of Sept. 2007) Prague, 2007. P. 63–70.
45. Jirina M., Jirina M. jr. Genetic Selection and Cloning in GMDH MIA Method. Proc. of
the II Intern. Workshop on Inductive Modelling IWIM 2007 (Prague, 23–26th of Sept.,
2007) Prague, 2007. P. 165–171.
46. Lytvynenko V., Bidyuk P., Myrgorod V. Application of the Method and Combined Algorithm
on the Basis of Immune Network and Negative Selection for Identification of Turbine Engine
Surging. Proc. of the II Intern. Conf. on Inductive Modelling ICIM-2008 (Kyiv, 15–19th
of Sept. 2008). Kyiv, 2008. P. 116–123.
47. Kordik P. Fully automated knowledge extraction using group of adaptive model evolu-
tion: PhD thesis. Prague: CTU, 2006. 150 р.
48. Oh S.K., Pedrycz W., Park H.S. Multi-layer hybrid fuzzy polynomial neural networks: a design
in the framework of computational intelligence. Neurocomputing. 2005. 64. P. 397–431.
49. Zaychenko Yu. The Investigations of Fuzzy Group Method of Data Handling with Fuzzy Inputs
in the Problem of Forecasting in Financial Sphere. Proc. of the II Intern. Conf. on Inductive
Modelling ICIM-2008. (Kyiv, 15–19th of Sept., 2008). Kyiv, 2008. P. 129–133.
50. Bodyanskiy Ye., Vynokurova O., Teslenko N. Cascade GMDH-Wavelet-Neuro-Fuzzy
Network. Proc. of the IV Intern. Workshop on Inductive Modelling IWIM-2011 (Kyiv-
Zhukyn, 4–11th of July, 2011). Kyiv-Zhukyn, 2011. P. 16–21.
51. Voytyuk I., Dyvak M., Spilchuk V. The Method of Structure Identification of Macromodels
as Difference Operators Based on the Analysis of Interval Data and Genetic Algorithm. Proc.
of the IV Intern. Workshop on Inductive Modelling IWIM-2011 (Kyiv-Zhukyn 4–11th of
July, 2011). Kyiv-Zhukyn, 2011. P. 114–118.
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 58
52. Lemke F. Parallel Self-Organizing Modeling. Proc. of the II Int. Conf. on Inductive
Modelling ICIM-2008 (Kyiv, 15-19th of Sept. 2008). Kyiv, 2008. P. 176–183.
53. Koshulko O.A., Koshulko A.I. Multistage combinatorial GMDH algorithm for parallel process-
ing of high-dimensional data. Proc. of III Int. Workshop on Inductive Modelling IWIM-
2009(15–19th of Sept., 2009, Krynica). Krynica, Poland, 2009. Prague: CTU, 2009. P. 114–116.
54. Kordík P., Černý J. Advanced Ensemble Strategies for Polynomial Models. Proc. of the
III Intern. Conf. on Inductive Modelling ICIM-2010 (Yevpatoria, 16–22nd of May,
2010). Yevpatoria, 2010. Kherson: KNTU, 2010. P. 77–82.
55. Čepek M., Kordík P., Šnorek M. The Effect of Modelling Method to the Inductive Preproc-
essing Algorithm. Proc. of the III Intern. Conf. on Inductive Modelling ICIM-2010. (Yevpa-
toria, 16–22nd of May 2010) Yevpatoria, 2010. Kherson: KNTU, 2010. P. 131–138.
56. Sarychev A.P. System Regularity Criterion of Group Method of Data Handling. Journal
of Automation and Information Sciences. 2006. 38(11). P. 25–37.
57. Sarycheva L. Quality Criteria for GMDH-based Clustering. Proc. of the II International Con-
ference on Inductive Modelling ICIM-2008 (Kyiv, 15–19th of Sept., 2008), Kyiv, 2008.
58. Lemke, F. Insights v.2.0, Self-organizing knowledge mining and forecasting tool,
2013.URL: http://www.knowledgeminer.eu. (Last accessed: 01.11.2018)
59. URL: https://www.gmdhshell.com. (Last accessed: 15.11.2018)
60. URL: www.mgua.irtc.org.ua (Last accessed: 01.12.2018)
61. Stepashko V.S. Method of Critical Variances as Analytical Tool of Theory of Inductive
Modeling. Journal of Automation and Information Sciences. 2008. 40(2). P. 4–22.
62. Ivakhnenko A.G., Savchenko E.A. Investigation of Efficiency of Additional Determination
Method of the Model Selection in the Modeling Problems by Application of the GMDH Al-
gorithm. Journal of Automation and Information sciences. 2008. 40(3). P. 47–58.
63. Stepashko V.S., Efimenko S.M. Sequential Estimation of the Parameters of Regression
Models. Cybernetics and Systems Analysis. 2005. 41(4). P.631–634.
64. Stepashko V., Yefimenko S. Parallel algorithms for solving combinatorial macromodel-
ling problems. Przegląd Elektrotechniczny (Electrical Review). 2009. 85(4). P 98–99.
65. Samoilenko O., Stepashko V. Method of Successive Elimination of Spurious Arguments
for Effective Solution the Search-Based Modelling Tasks. Proc. of the II Intern. Conf. on
Inductive Modelling ICIM-2008 (Kyiv,15–19th of Sept. 2008), Kyiv, 2008. P. 36–39.
66. Moroz O., Stepashko V. Hybrid sorting-out algorithm COMBI-GA with evolutionary
growth of model complexity. In: Advances in Intelligent Systems and Computing II:
Selected Papers from the International Conference on Computer Science and Informa-
tion Technologies, CSIT 2017, Lviv, Ukraine / N. Shakhovska, V. Stepashko, Ed. AISC,
Vol. 689. Cham: Springer, 2018. P. 346–360.
67. Stepashko V., Bulgakova O., Zosimov V. Construction and Research of the Generalized
Iterative GMDH Algorithm with Active Neurons. In: Advances in Intelligent Systems
and Computing II: Selected Papers from the International Conference on Computer Sci-
ence and Information Technologies, CSIT 2017, Lviv, Ukraine / N. Shakhovska, V. Ste-
pashko, Editors. AISC book series, Vol. 689. Cham: Springer, 2018. P. 492–510.
68. Pavlov A.V. Generalized relaxational iterative algorithm of GMDH. Inductive
Modeling of Complex Systems. Collected papers. Issue 3. Kyiv: IRTC ITS NASU,
2011. P. 121–134. (In Ukrainian)
69. Stepashko, V.S.: Conceptual fundamentals of intelligent modeling. Control Systems and
Machines (USiM). 2016. 4, pp. 3–15. (In Russian)
70. Yefimenko S.N., Stepashko V.S. Fundamentals of recurrent-and-parallel computing in
the combinatorial algorithm COMBI GMDH. USiM. 2014. 6. P. 27–33. (In Russian)
71. Yefimenko S.N., Stepashko V.S. Simulation experiment as a means of effectiveness re-
search of modeling methods from observation data. USiM. 2009. 1. P. 69–78. (In Russian)
72. Bulgakova O., Zosimov V., Stepashko V. Software package for modeling of complex
systems based on iterative GMDH algorithms with the network access capability. System
Research and Information Technologies. 1. 2014. P. 43–55. (In Ukrainian)
73. Pavlov A. Designing an automated structural-parametric identification system. Inductive
Modeling of Complex Systems. Collected papers. Issue 7. Kyiv: IRTC ITS NASU, 2015.
P. 202–219. (In Ukrainian)
Formation and Development of Self-Organizing Intelligent Technologies of Inductive
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 59
74. Yefimenko S. Building Vector Autoregressive Models Using COMBI GMDH with
Recurrent-and-Parallel Computations. In: Advances in Intelligent Systems and Comput-
ing II: Selected Papers from the International Conference on Computer Science and In-
formation Technologies, CSIT 2017, Lviv, Ukraine / N. Shakhovska, V. Stepashko, Edi-
tors. AISC book series, Vol. 689. Cham: Springer, 2018. P. 601–613.
75. Stepashko V., Samoilenko O., Voloschuk R. Informational Support of Managerial Deci-
sions as a New Kind of Business Intelligence Systems. In: Computational Models for
Business and Engineering Domains. G. Setlak, K. Markov (Eds.). Rzeszow, Poland; So-
fia, Bulgaria: ITHEA. 2014. P. 269–279.
76. Moroz O., Stepashko V. Data reconstruction of seasonal changes of amylolytic microor-
ganisms amount in copper polluted soils. Proc. of the 13th IEEE Intern. Conf. CSIT-2018
& International Workshop on Inductive Modeling. (Lviv, 11–14th of Sept., 2018), Lviv,
2018. P. 479–482.
77. Stepashko V.S., Yefimenko S.M., Savchenko Ye.A. Computerized experiment in induc-
tive modeling. Kyiv: Naukova Dumka, 2014. 222 p. (In Ukrainian)
78. Pavlov A.V., Stepashko V.S., Kondrashova N.V. Effective methods of models self-
organization. Kyiv: Akademperiodika, 2014. 200 p. (In Russian)
79. Stepashko V.S., Bulgakova O., Zosimov V. Iterational algorithms of inductive modeling.
Kyiv: Naukova Dumka, 2014. 190 p. (In Ukrainian)
80. Schmidhuber J. Deep learning in neural networks: An overview. Neural Networks. 2015.
61, pp. 85–117.
Received 20.09.2018
Степашко В.С., д-р техн. наук, професор
зав. відд. інформаційних технологій індуктивного моделювання,
e-mail: stepashko@irtc.org.ua
Міжнародний науково-навчальний центр інформаційних технологій
та систем НАН України та МОН України,
пр. Акад. Глушкова, 40, м. Київ, 03187, Україна
ФОРМУВАННЯ I РОЗВИТОК САМООРГАНІЗОВНИХ ІНТЕЛЕКТУАЛЬНИХ
ТЕХНОЛОГІЙ ІНДУКТИВНОГО МОДЕЛЮВАННЯ
Вступ. Ефективне розв’язання завдань керування та прийняття рішень у складних
системах має використовувати результати математичного моделювання. Для побудови
адекватних прогнозних моделей є багато сучасних методів та інструментів, які, як
правило, базуються на двох основних підходах: керовані теорією (дедуктивні) та керо-
вані даними (індуктивні). Методи, керовані даними, є основними для розв’язання
типових задач аналізу даних; вони реалізують індуктивний процес переходу від конк-
ретних даних до моделей, що узагальнюють ці дані. Серед усіх таких методів досить
примітними є ті, що розробляються в рамках індуктивного моделювання на основі
методу групового урахування аргументів (МГУА), створеного кілька десятиліть тому
академіком О.Г. Івахненком.
Метою дослідження є аналіз передумов винайдення МГУА О.Г. Івахненком та
еволюції ідей, методів та інструментів самоорганізації моделей протягом піввікового
історичного періоду успішного розвитку методології індуктивного моделювання.
Результати. Знання, набуті О.Г. Івахненком у галузях автоматичного керування, ін-
женерної кібернетики та зароджуваної нейронауки, ініційованої ідеями персептрона, були
тими передумовами, які допомогли йому синтезувати оригінальний самоорганізовний
підхід до розв’язання завдань побудови моделей об'єктів і процесів на основі експеримен-
тальних даних. У роботі проаналізовано еволюцію наукових ідей та основні досягнення у
розвитку МГУА в період 1968–1997 років. Охарактеризовано внесок науковців з різних
країн у модифікацію та застосування МГУА. Наведено результати подальшого розроблен-
ня методів та інструментів індуктивного моделювання у відділі інформаційних технологій
індуктивного моделювання і вказано найперспективніші напрями досліджень у цій галузі.
Stepashko V.S.
ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 60
Висновки. Проаналізовано основні передумови, що сприяли створенню МГУА
О.Г. Івахненком, охарактеризовано основні фундаментальні, технологічні та прикладні
досягнення півстолітнього розвитку індуктивного моделювання як в Україні, так і за
кордоном, а також сформульовано найбільш перспективні шляхи подальших досліджень.
Ключові слова: математичне моделювання, моделювання на основі даних, самооргані-
зація моделі, МГУА, індуктивне моделювання, завадостійке моделювання, інформацій-
ні технології, прикладне дослідження.
Степашко В.С., д-р техн. наук, профессор,
зав. отд. информационных технологий индуктивного моделирования,
e-mail: stepashko@irtc.org.ua
Международный научно-учебный центр информационных
технологий и систем НАН Украины и МОН Украины,
пр. Акад. Глушкова, 40, г. Киев, 03187, Украина
ФОРМИРОВАНИЕ И РАЗВИТИЕ САМООРГАНИЗУЮЩИХСЯ
ИНТЕЛЛЕКТУАЛЬНЫХ ТЕХНОЛОГИЙ ИНДУКТИВНОГО МОДЕЛИРОВАНИЯ
Целью работы является анализ эволюции идей, методов и инструментов самоорганизации
моделей в течение полувекового исторического периода успешного развития методологии
индуктивного моделирования. Проанализированы основные предпосылки создания академи-
ком А.Г. Ивахненко метода группового учета аргументов (МГУА), исследуется эволюция его
научных идей и взглядов, а также основные достижения в развитии МГУА в период
1968–1997 годов. Охарактеризован вклад исследователей разных стран в модификацию и
применение МГУА. Приведены результаты дальнейших разработок методов и инструментов
индуктивного моделирования в отделе Информационных технологий индуктивного модели-
рования и указаны наиболее перспективные направления исследований в этой области.
Ключевые слова: математическое моделирование, моделирование на основе данных,
самоорганизация модели, МГУА, индуктивное моделирование, помехоустойчивое мо-
делирование, информационные технологии, прикладное исследование.
<<
/ASCII85EncodePages false
/AllowTransparency false
/AutoPositionEPSFiles true
/AutoRotatePages /All
/Binding /Left
/CalGrayProfile (Dot Gain 20%)
/CalRGBProfile (sRGB IEC61966-2.1)
/CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2)
/sRGBProfile (sRGB IEC61966-2.1)
/CannotEmbedFontPolicy /Warning
/CompatibilityLevel 1.4
/CompressObjects /Tags
/CompressPages true
/ConvertImagesToIndexed true
/PassThroughJPEGImages true
/CreateJDFFile false
/CreateJobTicket false
/DefaultRenderingIntent /Default
/DetectBlends true
/DetectCurves 0.0000
/ColorConversionStrategy /LeaveColorUnchanged
/DoThumbnails false
/EmbedAllFonts true
/EmbedOpenType false
/ParseICCProfilesInComments true
/EmbedJobOptions true
/DSCReportingLevel 0
/EmitDSCWarnings false
/EndPage -1
/ImageMemory 1048576
/LockDistillerParams false
/MaxSubsetPct 100
/Optimize true
/OPM 1
/ParseDSCComments true
/ParseDSCCommentsForDocInfo true
/PreserveCopyPage true
/PreserveDICMYKValues true
/PreserveEPSInfo true
/PreserveFlatness true
/PreserveHalftoneInfo false
/PreserveOPIComments false
/PreserveOverprintSettings true
/StartPage 1
/SubsetFonts true
/TransferFunctionInfo /Apply
/UCRandBGInfo /Preserve
/UsePrologue false
/ColorSettingsFile ()
/AlwaysEmbed [ true
]
/NeverEmbed [ true
]
/AntiAliasColorImages false
/CropColorImages true
/ColorImageMinResolution 300
/ColorImageMinResolutionPolicy /OK
/DownsampleColorImages true
/ColorImageDownsampleType /Bicubic
/ColorImageResolution 300
/ColorImageDepth -1
/ColorImageMinDownsampleDepth 1
/ColorImageDownsampleThreshold 1.50000
/EncodeColorImages true
/ColorImageFilter /DCTEncode
/AutoFilterColorImages true
/ColorImageAutoFilterStrategy /JPEG
/ColorACSImageDict <<
/QFactor 0.15
/HSamples [1 1 1 1] /VSamples [1 1 1 1]
>>
/ColorImageDict <<
/QFactor 0.15
/HSamples [1 1 1 1] /VSamples [1 1 1 1]
>>
/JPEG2000ColorACSImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/JPEG2000ColorImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/AntiAliasGrayImages false
/CropGrayImages true
/GrayImageMinResolution 300
/GrayImageMinResolutionPolicy /OK
/DownsampleGrayImages true
/GrayImageDownsampleType /Bicubic
/GrayImageResolution 300
/GrayImageDepth -1
/GrayImageMinDownsampleDepth 2
/GrayImageDownsampleThreshold 1.50000
/EncodeGrayImages true
/GrayImageFilter /DCTEncode
/AutoFilterGrayImages true
/GrayImageAutoFilterStrategy /JPEG
/GrayACSImageDict <<
/QFactor 0.15
/HSamples [1 1 1 1] /VSamples [1 1 1 1]
>>
/GrayImageDict <<
/QFactor 0.15
/HSamples [1 1 1 1] /VSamples [1 1 1 1]
>>
/JPEG2000GrayACSImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/JPEG2000GrayImageDict <<
/TileWidth 256
/TileHeight 256
/Quality 30
>>
/AntiAliasMonoImages false
/CropMonoImages true
/MonoImageMinResolution 1200
/MonoImageMinResolutionPolicy /OK
/DownsampleMonoImages true
/MonoImageDownsampleType /Bicubic
/MonoImageResolution 1200
/MonoImageDepth -1
/MonoImageDownsampleThreshold 1.50000
/EncodeMonoImages true
/MonoImageFilter /CCITTFaxEncode
/MonoImageDict <<
/K -1
>>
/AllowPSXObjects false
/CheckCompliance [
/None
]
/PDFX1aCheck false
/PDFX3Check false
/PDFXCompliantPDFOnly false
/PDFXNoTrimBoxError true
/PDFXTrimBoxToMediaBoxOffset [
0.00000
0.00000
0.00000
0.00000
]
/PDFXSetBleedBoxToMediaBox true
/PDFXBleedBoxToTrimBoxOffset [
0.00000
0.00000
0.00000
0.00000
]
/PDFXOutputIntentProfile ()
/PDFXOutputConditionIdentifier ()
/PDFXOutputCondition ()
/PDFXRegistryName ()
/PDFXTrapped /False
/Description <<
/CHS <FEFF4f7f75288fd94e9b8bbe5b9a521b5efa7684002000500044004600206587686353ef901a8fc7684c976262535370673a548c002000700072006f006f00660065007200208fdb884c9ad88d2891cf62535370300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002>
/CHT <FEFF4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef653ef5728684c9762537088686a5f548c002000700072006f006f00660065007200204e0a73725f979ad854c18cea7684521753706548679c300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002>
/DAN <FEFF004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002000740069006c0020006b00760061006c00690074006500740073007500640073006b007200690076006e0069006e006700200065006c006c006500720020006b006f007200720065006b007400750072006c00e60073006e0069006e0067002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e>
/DEU <FEFF00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f00630068007700650072007400690067006500200044007200750063006b006500200061007500660020004400650073006b0074006f0070002d0044007200750063006b00650072006e00200075006e0064002000500072006f006f0066002d00470065007200e400740065006e002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e>
/ESP <FEFF005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f0073002000640065002000410064006f0062006500200050004400460020007000610072006100200063006f006e00730065006700750069007200200069006d0070007200650073006900f3006e002000640065002000630061006c006900640061006400200065006e00200069006d0070007200650073006f0072006100730020006400650020006500730063007200690074006f00720069006f00200079002000680065007200720061006d00690065006e00740061007300200064006500200063006f00720072006500630063006900f3006e002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e>
/FRA <FEFF005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f007500720020006400650073002000e90070007200650075007600650073002000650074002000640065007300200069006d007000720065007300730069006f006e00730020006400650020006800610075007400650020007100750061006c0069007400e90020007300750072002000640065007300200069006d007000720069006d0061006e0074006500730020006400650020006200750072006500610075002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e>
/ITA <FEFF005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f006200650020005000440046002000700065007200200075006e00610020007300740061006d007000610020006400690020007100750061006c0069007400e00020007300750020007300740061006d00700061006e0074006900200065002000700072006f006f0066006500720020006400650073006b0074006f0070002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e>
/JPN <FEFF9ad854c18cea51fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e30593002537052376642306e753b8cea3092670059279650306b4fdd306430533068304c3067304d307e3059300230c730b930af30c830c330d730d730ea30f330bf3067306e53705237307e305f306f30d730eb30fc30d57528306b9069305730663044307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e30593002>
/KOR <FEFFc7740020c124c815c7440020c0acc6a9d558c5ec0020b370c2a4d06cd0d10020d504b9b0d1300020bc0f0020ad50c815ae30c5d0c11c0020ace0d488c9c8b85c0020c778c1c4d560002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e>
/NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken voor kwaliteitsafdrukken op desktopprinters en proofers. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.)
/NOR <FEFF004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200066006f00720020007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c00690074006500740020007000e500200062006f007200640073006b0072006900760065007200200065006c006c00650072002000700072006f006f006600650072002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e>
/PTB <FEFF005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020007000610072006100200069006d0070007200650073007300f5006500730020006400650020007100750061006c0069006400610064006500200065006d00200069006d00700072006500730073006f0072006100730020006400650073006b0074006f00700020006500200064006900730070006f00730069007400690076006f0073002000640065002000700072006f00760061002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e>
/SUO <FEFF004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f0074002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a00610020006c0061006100640075006b006100730074006100200074007900f6007000f60079007400e400740075006c006f0073007400750073007400610020006a00610020007600650064006f007300740075007300740061002000760061007200740065006e002e00200020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e>
/SVE <FEFF0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740020006600f600720020006b00760061006c00690074006500740073007500740073006b0072006900660074006500720020007000e5002000760061006e006c00690067006100200073006b0072006900760061007200650020006f006300680020006600f600720020006b006f007200720065006b007400750072002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e>
/ENU (Use these settings to create Adobe PDF documents for quality printing on desktop printers and proofers. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.)
>>
/Namespace [
(Adobe)
(Common)
(1.0)
]
/OtherNamespaces [
<<
/AsReaderSpreads false
/CropImagesToFrames true
/ErrorControl /WarnAndContinue
/FlattenerIgnoreSpreadOverrides false
/IncludeGuidesGrids false
/IncludeNonPrinting false
/IncludeSlug false
/Namespace [
(Adobe)
(InDesign)
(4.0)
]
/OmitPlacedBitmaps false
/OmitPlacedEPS false
/OmitPlacedPDF false
/SimulateOverprint /Legacy
>>
<<
/AddBleedMarks false
/AddColorBars false
/AddCropMarks false
/AddPageInfo false
/AddRegMarks false
/ConvertColors /NoConversion
/DestinationProfileName ()
/DestinationProfileSelector /NA
/Downsample16BitImages true
/FlattenerPreset <<
/PresetSelector /MediumResolution
>>
/FormElements false
/GenerateStructure true
/IncludeBookmarks false
/IncludeHyperlinks false
/IncludeInteractive false
/IncludeLayers false
/IncludeProfiles true
/MultimediaHandling /UseObjectSettings
/Namespace [
(Adobe)
(CreativeSuite)
(2.0)
]
/PDFXOutputIntentProfileSelector /NA
/PreserveEditing true
/UntaggedCMYKHandling /LeaveUntagged
/UntaggedRGBHandling /LeaveUntagged
/UseDocumentBleed false
>>
]
>> setdistillerparams
<<
/HWResolution [2400 2400]
/PageSize [612.000 792.000]
>> setpagedevice
|