Paradigm of nonstochastic approach tosystem identification

The paper discusses the construction of linear models, the complexity of which is determined by dimensionality. Within the framework of the non-stochastic approach, a methodological and mathematical foundation for reconstructing models that describe processes in complex systems is developed. Asympto...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Проблеми керування та інформатики
Datum:2023
Hauptverfasser: Gubarev, V., Melnychuk, S., Salnikov, N.
Format: Artikel
Sprache:Ukrainisch
Veröffentlicht: Інститут кібернетики ім. В.М. Глушкова НАН України 2023
Schlagworte:
Online Zugang:https://nasplib.isofts.kiev.ua/handle/123456789/210935
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Назва журналу:Digital Library of Periodicals of National Academy of Sciences of Ukraine
Zitieren:Paradigm of nonstochastic approach tosystem identification / V. Gubarev, S. Melnychuk, N. Salnikov // Проблеми керування та інформатики. — 2023. — № 1. — С. 42–58. — Бібліогр.: 32 назв. — англ.

Institution

Digital Library of Periodicals of National Academy of Sciences of Ukraine
_version_ 1859506160690790400
author Gubarev, V.
Melnychuk, S.
Salnikov, N.
author_facet Gubarev, V.
Melnychuk, S.
Salnikov, N.
citation_txt Paradigm of nonstochastic approach tosystem identification / V. Gubarev, S. Melnychuk, N. Salnikov // Проблеми керування та інформатики. — 2023. — № 1. — С. 42–58. — Бібліогр.: 32 назв. — англ.
collection DSpace DC
container_title Проблеми керування та інформатики
description The paper discusses the construction of linear models, the complexity of which is determined by dimensionality. Within the framework of the non-stochastic approach, a methodological and mathematical foundation for reconstructing models that describe processes in complex systems is developed. Asymptotic modeling allows for the formation of classes of models suitable for solving identification problems. An exact description corresponds to an infinite extension, and thus the quality of the model improves with its increasing dimensionality. However, errors in the available data prevent the indefinite increase in dimensionality due to the ill-posedness of the identification problem starting from a certain dimension. The regularization procedure enables the determination of an effective approximate solution to the identification problem, which, in the non-stochastic case, is consistent with the data errors. The properties and features of the proposed approach are illustrated by modeling results. У статті розглядається побудова лінійних моделей, складність яких визначається розмірністю. У рамках нестохастичного підходу розроблено методологічну та математичну основу реконструкції моделей, що описують процеси у складних системах. Асимптотичне моделювання доз­воляє для такої системи формувати класи моделей, які підходять для розв’язання задачі ідентифікації. Точний опис відповідає нескінченному розширенню, тому якість моделі покращується зі збільшенням її роз­мірності. Однак помилки в наявних даних не дозволяють безмежно збільшувати їхню розмірність через погану обумовленість задачі ідентифікації, починаючи з деякого виміру. Процедура регуляризації дозволяє визначити ефективне наближене рішення задачі ідентифікації, яке для нестохастичного випадку узгоджується з помилками даних. Властивості та особливості пропонованого підходу ілюструються результатами моделювання.
first_indexed 2026-03-13T00:52:50Z
format Article
fulltext © V. GUBAREV, S. MELNYCHUK, N. SALNIKOV, 2023 42 ISSN 2786-6491 МЕТОДИ КЕРУВАННЯ ТА ОЦІНЮВАННЯ В УМОВАХ НЕВИЗНАЧЕНОСТІ UDС 681.5.015; 519.233 V. Gubarev, S. Melnychuk, N. Salnikov PARADIGM OF NONSTOCHASTIC APPROACH TO SYSTEM IDENTIFICATION Vyacheslav Gubarev Space Research Institute of NASU and SSAU, Kyiv, orcid: 0000-0001-6284-1866 v.f.gubarev@gmail.com Serhii Melnychuk Space Research Institute of NASU and SSAU, Kyiv, orcid: 0000-0002-6027-7613 serg.vik@ukr.net Nikolay Salnikov Space Research Institute of NASU and SSAU, Kyiv, orcid: 0000-0001-9810-0963 salnikov.nikolai@gmail.com The concept of a complex system in this work is understood as a large set of dynamic interconnected systems, the exact mathematical model of which is not known or has a very large dimension. In such situation the use of standard methods for synthesizing feedback becomes difficult or even im- possible due to the degeneracy of the corresponding mathematical prob- lems. One way out of this situation is to build an approximation model of reduced dimension. This can be done using a system of initial equations , if they are available, or using identification methods based on measurements of output and input variables acting on the system. In this case, the process of constructing a mathematical model is reduced to a sequential enumeration of possible models of increasing complexity. As a criterion for the adequacy of the model, the norm of deviation of the output of the adjusted model from the measured value of the output of the system under study is considered. The article deals with the construction of linear models, the complexity of which is determined by their dimension. In the framework of nonstochastic approach it is developed the methodological and mathematical basis for model reconstruction which describes processes in complex systems. As- ymptotic modelling allows for such system to form model classes appropri- ate to solve identification problem. Precise description corresponds to infi- nite expansion so the model quality is improved when its dimension is in-  S. Melnychuk is grateful to the National Research Fund of Ukraine for support (project 2020.02/0015 «Theoretical and experimental research of global disturbances of natural and man-made origin in the Earth-Atmosphere-Ionosphere system»). mailto:v.f.gubarev@gmail.com mailto:serg.vik@ukr.net mailto:salnikov.nikolai@gmail.com Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 43 creased. However errors in available data do not allow increase their dimen- sion limitlessly due to ill-conditionality of the identification problem begin- ning from some dimension. Regularization procedure permits to determine the effective approximate solution of identification problem which for non- stochastic case is in agreement with errors in data. Properties and peculiari- ties of the proposed approach are illustrated by simulation results. Keywords: system identification, linear regression, regularization, asymptotic modelling, approximate solution Introduction System identification, that is, the construction of a mathematical model of a system based on data obtained as a result of experiments, is primarily focused on complex dy- namic processes for which it is impossible to establish the laws of their behaviour in other ways. This means that even in a general form it is impossible to indicate a class of mathematical models containing an exact description of the system under study. Here we consider a new concept or paradigm of system identification within a non- stochastic approach for linear time invariant (LTI) systems. We shell assume the exist- ence of abstract transfer function matrix ( )G z including nonrational cases, which con- nects the input and the measured output. According to the concept of asymptotic model- ling, widely used in computational mathematics, an unknown transfer function can be written as finite or infinite expansions for some basis functions, which makes it possible to write a class of models in a form convenient for identification. At such approach, we are talking about the construction of finite-dimensional approxi- mating models. At the same time, with an increase in the dimension, the accuracy of the de- scription of the system increases if certain requirements for the system under study are met. For example, truncated rational approximation of infinite-dimensional LTI system with fi- nite-dimensional input and output that induced nuclear type Hankel operator with distinct singular values convergences to precise model as it dimension tends to infinity [1]. Asymptotically stable system can be written as an infinity expansion 0 ( ) ( )k k k G z G f z     (1) where 0,1,2,{ ( )}k kf z  is a sequence of orthonormal functions and 0,1,2,{ }k kG  is a sequence of parameters. Then task of identification is to find a finite-dimensional model 1 0 ( ) ( ). n k k k G z G f z     (2) The accuracy of the model depend on the choice of basis functions and dimen- sion .n The case of k kf z corresponds to Infinite Impulse Response (IIR) mode- ling [2]. In some cases the use of Laguerre or Kautz polynomials gives a better approx- imation [3, 4]. In general, the advantage in identification and approximation can be achieved by using such orthonormal basis functions that correspond to dynamics close to the dynamics of the object under study [5]. A very popular form for rational approximation is the Linear Regression which di- rectly links input and output variables [2]. Here, for ease of paradigm presentation, we consider an ARX model with a scalar input ( )u t and output ( )y t 1 1( ) ( 1) ( ) ( 1) ( ) a bn a n by t a y t a y t n b u t b u t n          (3) in discrete time 0, 1,t  . Let us denote vectors T 1[ , , ] , aa na a  T 1[ , , ] , bb nb b  T( ) [ ( 1), , ( )]a at y t y t n      and T( ) [ ( 1), , ( )] ,b bt u t u t n    so (3) can be written as follows T T( ) ( ) ( ) .a a b by t t t     (4) 44 ISSN 2786-6491 The special case 0an  gives us Finite Impulse Response (FIR) approximate mo- del. Any ARX model has an equivalent state-space model representation, so it is easy to pass from one description to another using linear transformation. It is important because the trun- cated state-space models are approximating for nuclear type nonrational system [1]. 1. Paradigm of stochastic identification Let us briefly consider identification paradigm in stochastic case according to [6]. Mathematical difficulties in system identification are associated with the presence of uncertainty in the data. The generally accepted is the stochastic interpretation of uncer- tainty, which assumes that measurement errors are independent and identically distrib- uted (i.i.d.) random variables. This paradigm underlies all classical statistical methods in system identification. Model parameters of a given order are estimated by maximum likelihood methods (prediction errors), which in most cases are formulated as extremal problems 2 1 ˆ ˆarg min ( ) ( / ) , N D t y t y t       (5) where ( )y t is a measured output, ˆ( / )y t  is a prediction by the model, D is a set of values . There are many publications devoted to the identification problem in such statement, see, for example [2, 7, 8]. Under concept of asymptotic modeling the key question in identification is a model order determination. It is known that a higher order model can better approximate the measured output of the system, i.e. reduce the error called «bias». On the other hand, a higher-order model is more sensitive to errors in the data, which corresponds to a larger variance in model parameter estimates. The mismatch between system and model in- cludes both of these components. Traditional system identification often uses the bias- variance trade-off to minimize the total mean square error (MSE). Various procedures have been proposed for this, among which are the Akaikeʼs Information Criteria (AIC) and Bayesian Information Criteria (BIC) order criteria described in the books men- tioned above. 2. Nonstochastic approach to EIV-identification In many cases on practice, errors occur at output and input variables measurement. Such situation in system identification is called «errors in variables» (EIV) and was considered by many authors. Results obtained in the framework of stochastic paradigm are described in [9–13]. Recently EIV identification in frequency domain was consid- ered in [14]. New results also presented in [15–17]. This article develops an alternative approach based on the non-stochastic para- digm. We assume that data errors are random variables belonging to known bounded sets with arbitrary distributions, including worst cases. With this formulation, the max- imum likelihood method is not appropriate. Early the similar approach was implement- ed in so-called set membership identification (SMI) in which the main goal was to de- termine a guaranteed set of models, including the exact one. The description of these methods is given in [18–21]. The approach developing in this article is an alternative to SMI and aims to find a single approximate solution of identification problem consistent with errors in availa- ble data. We have the following measurements of true values ( )u t and ( )y t ( ) ( ) ( ),uu t u t t  ( ) ( ) ( ),yy t y t t  (6) Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 45 where ( )u t and ( )y t are errors. According to the mentioned assumption ( )u t and ( )y t are unknown arbitrary random sequences satisfying conditions ( ) ,u ut   ( ) .y yt   (7) An important issue in identification is the informative input that excites all modes of the system. In stochastic methods, a persistent excitation signal of a given order is usually used for this. In the case of non-stochastic identification, we propose an active experiment in which each mode of the system at a certain moment of observation makes the maximum possible contribution to the output. For an asymptotically stable system, this can be achieved by two ways. The first is to collect data from separate experiments consisting of the intervals of excitation and following relaxation. An alternative is a sin- gle continuous experiment where excitation intervals alternate with relaxations. In both cases, the duration of relaxation should exceed the transient time. Consider an observable and controllable discrete LTI system. Such a system can be represented by linear regression or its state-space equivalent ( 1) ( ) ( ),x t Ax t bu t   T( ) ( ) ( )y t c x t du t  (8) where t is a discrete time and «T» denotes a transposition. Assume that this system does not have multiple eigenvalues. Then we can choose the Jordan block realization: diag ( ),pA A col( ),pb b col( ),pc c where real eigenvalues p p   corre- spond to blocks ,p pA   ,c p pb b ,c p pc c (9) and complex eigenvalues p p pi     correspond to blocks p p p p p A          , c p p s p b b b            , c p p s p c c c            . (10) In case of finite-dimensional system with ( ) 0y t  at 0,t  ( ) 0u t  at 0t  and ( ) 0u t  at 0t  input-output ration (8)–(10) is equivalent to 1 0 0 1 ( ) ( ) ( ), t P p j p y t h t j u j        (11) where 0 ( ) [ cos sin ],k c s p p p p p ph k f k f k    p p   , arg ,p p   c pf  ,c c s s p p p pc b c b  .s c s s c p p p p pf c b c b  Parameters ,p ,p ,c pf s pf completely deter- mine the system dynamics. In case of real eigenvalues we have ,p p   0,p  0.s s p pc b  In EIV identification the informativeness of the input is defined by the signal-to- noise ratio (SNR). The larger SNR of the output for each mode (10), the more informa- tive the input is. Therefore, we will shape the input so that at the end of the excitation interval, a certain mode has the largest output signal. Different ( )u t across experiments should effectively excite each of the modes. Two types of input signal will be used for this. The first is a rectangular pulse 46 ISSN 2786-6491 1, 1, ( ) 0, else it u t       (12) where i is of a varied duration across experiments: ,i i  max1, 2, , .i i Short rec- tangular pulses provide stronger excitation of fast modes. Increasing the duration will add a contribution from slow modes. Therefore a set of rectangular pulses of different duration should provide separate excitation of individual modes corresponding to real eigenvalues (9). The second type of input is a single harmonic sin , 1, ( ) 0, else jt t u t        (13) where frequency j change across experiments ,j j   max1, 2, ,j j and cover the range (0, π / 2). For modes corresponding to complex eigenvalues (10), we aim to catch resonant excitation when j happens to be close to the natural frequency .p When j are varied with a small step, it is possible to provide resonant excitation of all oscillating modes of the system. In this case the duration  is chosen to be sufficient to establish steady-state forced oscillations. With an appropriate choice of parameters max ,i max ,j  and  it is possible to collect informative data with acceptable SNR with a fairly general knowledge of the system under study. More detailed information on this can be found in [22]. 3. Method of identification Here will be developed the identification method in the framework of nonstohastic approach. The goal is to obtaine a regularized solution that gives an approximation with an accuracy corresponding to the noise in data. As mentioned, the asymptotic class of linear autoregressive models is considered to identify complex systems that in this class have a large or infinite dimension. Conception of approximate regularized solution means that the model dimension may be less then the order of the true system. Therefore, identification must include the choice of model order. 3.1. Model order selection. The type of control signals proposed above makes it possible to divide the identification process into independent tasks. Measurements on relaxation intervals are used to determine the vector a and its dimension. For this we will use the equation 1 2( ) ( 1) ( 2) ( ),ny t a y t a y t a y t n        (14) which describe free motion on relaxation intervals. Applying controls (12) and (13), we collect the corresponding outputs into the following matrix: pulse relax harm relax pulse pulse pulse 1relax harm harm harm relax 2 , col ([ ( 1) ( ), 1, , col ([ ( 1) ( )]), 1, , i ii i j j Y Y Y Y y y l i k Y y y l j k                    (15) where pulse iy is a measured response on the i -th rectangular pulse (12), 1k is a total number of rectangular pulses, harm jy is a measured response on the j-th harmonic in- Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 47 put (13), 2k is a total number of input harmonics and l is the relaxation duration, the same for all experiments. We denote the total number of rows of Y by 1 2.k k k  Value l should be more then transient time and both k and l must exceed supposed model order. Let us produce singular value decomposition (SVD) of the matrix Y T,Y U V  (16) where ,U , V are matrices of dimensions ,k k k l and l l respectively. In the deterministic case for a finite dimensional system the rank of Y is equal to true dimension n and only the first n singular numbers 1, , n  are non-zero. Then the dimension of the system can be precisely determined by the number of non-zero di- agonal elements of . In the non-deterministic case the matrix Y almost always will be of full rank and all singular numbers will be positive. Even then the true dimension n can sometimes be determined by singular values, if there is a gap between the values n and 1.n How- ever there is often no clear gap. Result depends on the SNR of individual modes. Modes of the system with a low SNR becomes indistinguishable from background noise, mak- ing impossible in the non-stochastic case to establish the true dimension. In such cases, we can find a model of reduced dimension applying the known in regularization princi- ple that approximate solution should be consistent in accuracy with the data errors. For any chosen value n̂ we can divide the measured output into «signal» and «noise» part using a low-rank approximation in Frobenius norm. For this we split (16) T T0 [ ] [ ] , 0 S S N S N N U V U U V V         (17) where S contains n̂ largest singular values. Indices «S» and «N» denote «signal» and «noise» part. As a result we get decomposition ˆ ˆ ,S N n n Y Y Y  (18) where T ˆ ( )S S S S n Y U V  and T ˆ ( ) .N N N N n Y U V  Expansion (18) allows one to find the matrix of the given rank closest to full-rank matrix Y in Frobenius norm using  -rank property [23]. So we have ˆ ˆrank( )S n Y n and ˆ 1 ˆ ,N n n F Y  (19) where subscript «F» means Frobenius norm, defined as 2 , iji jF A a  for any ma- trix ( ).ijA a Matrix ˆ S n Y will be considered as the output of the desired model. Again we represent the matrix Y in the form ,Y Y Y  (20) where Y corresponds to noise-free output generated by true system, and Y corre- sponds to the noise, which, according to condition (7), is bounded in the infinity norm .yY     (21) Since all available information about Y is limited only by condition (21), then, ac- cording to the guaranteed approach, any Y  that satisfies the inequality Y Y      48 ISSN 2786-6491 can be considered as corresponding to a noise-free output of the exact model. Among all exact models, we are looking for a model of minimal order, which corresponds to the min- imal rank of the matrix .Y  Using decomposition (18) we determine the model by its out- put as ˆ ,S n Y increasing the dimension n̂ until condition ˆ S yn Y Y     is satisfied. From here comes the criterion ˆ ˆ 1 .N N yn n Y Y      (22) Using equivalence of norms , F A A A k l      which is valid for an arbi- trary matrix ,k lA R  from (22) and (19) we derive inequality ˆ 1 ˆ ,n y n kl      or ˆ ˆ 1 , . n y n y kl        (23) The condition (23) can be used to determine the dimension of the model that is consistent in accuracy with the errors. 3.2. Parametric identification. After determining the order ˆan n of vector ,a it can be found from the basic equations (3), (4), where ( ) 0.b t u   For this we construct an overdetermined linear system ,A a AW w  (24) where the matrix AW and vector Aw are formed from the matrix a S n Y as follows:     1 2 ( )a A A A l n A W W W W                     , (1) (2) ( )a A A A l n A w w w w                   . Here, (1) AW contains the first an columns of the matrix , a S n Y the matrix (2) AW also contains an columns, starting from the second, and so on. The vector (1) Aw is the ( 1)an  –th column of , a S n Y (2) Aw is the ( 2)an  –th column, etc. The last one vector ( )al n Aw  is the first column of the matrix . a N n Y System (24) is a strongly overdetermined for large r and .l Then we can discard non- informative equations with small SNR, i.e. having max / yy  less than some threshold, where maxy is the maximum modulus of regressors. As a result, we obtain a truncated but still overdetermined system .A a AW w  (25) The solution of (25) can be found by ordinary least squares (OLS) or by total least squares (TLS) [23]. However, in practice, very often problem (25) turns out to be ill- posed, having an ill-conditioned matrix .AW Therefore, it is proposed to implement the LS based on the SVD decomposition of the matrix anY T 1 1 1 .AW U V  (26) Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 49 Substituting (26) into (25), we obtain 1 1 ,Aw   (27) where T 1 1 ,aV   T 1 .A Aw U w  The ratio of the first singular value of the matrix 1 to the last one determines the conditionality of the problem. Under good conditionality, the least squares solution of (26) is 1 T 1 1 1 .a AV U w   (28) When the matrix AW is ill-conditioned, one should find a regularized solution, the construction of which is given below. At the last stage of identification, the dimension bn and coefficients of the vector b are determined. It is preceded by extracting data corresponding to a purely forced part of model. With an estimated vector ,a such a signal ( )y t for the moment t is determined by the relation 1 2( ) ( ) ( 1) ( 2) ( ).n ay t y t a y t a y t a y t n        (29) Then the basic equation for finding the vector b is 1( ) ( 1) ( ),m by t b u t b u t n     (30) where dimension bn of the vector b in the main not exceed dimension of vector .a In principle, case b an n is possible if there is a static connection between input and output, caused by very fast modes having a transient process shorter than the sampling step. After obtaining ( )y t from (29), we select samples corresponding to excitation in- tervals defined by (12) and (13) and form an overdetermined system ,B b BW w  (31) similar to the construction of AW and using .b an n Elements of the matrix BW are measured control signals ( ).u t The chosen method of excitation (12), (13) provides good conditionality of the matrix ,BW so system (31) can be solved by TLS. This completes the solution of the identification problem. As a result we obtain an approximate mathematical model, consistent in accuracy with the errors in the data de- termined by conditions (6), (7). 4. Ill-conditioning The solvability of the identification problem in the formulation under consideration is determined by the properties of the matrices AW in (24) and BW in (31), and, first of all, by their condition number. It was proposed to solve (24) and (31) using instead of OLS, a com- pletely equivalent method based on the SVD decomposition of these matrices. Then the con- dition number of these matrices with respect to the norm 2  can be found from the relation 1 , n     (32) where ,a bn n n correspond to the dimensions of the allocated blocks when choosing the order of the model. Due to the properties of the SVD decomposition, as the 50 ISSN 2786-6491 number n increases, the number  can only grow. The growth nature of ( )n of matrices AW and BW is usually different, and each of them depends on its own factors. Since the matrix BW is formed from input signals, it is possible to control and influence behavior of ( )bn in active experiments. The method of excitation with alternating in- tervals of excitation and relaxation makes it possible to provide a rather weak change of ( )bn with increasing .bn Moreover, there is every reason to believe, that with values of ( )bn remaining close to unity, we will have the most informative signal. The behavior of condition number of the matrix AW depends on the following fac- tors. What remains is its dependence on the informative input action, i.e. its ability to excite all modes of the system with an acceptable SNR. The SNR indicator, which is a relative value, naturally includes all errors acting on the system. This equally applies to both matrices AW and .BW The value of ( )an significantly depends on the dynamic features of the system, determined by the values of the invariants p and ,p as well as c pf and .s pf There is a rapid increase in the condition number with increase of n when the eigenvalues of the system are clustered or when there are very fast modes in the system. Small values of c pf and s pf lead to small output signals of the correspond- ing modes, so that for them the SNR becomes of the order of unity. Nevertheless, even for systems with the most favorable values of these invariants and with an informative input, the condition number of matrix AW grows rather quickly as an increases, which is clearly seen from the Table. Table an 10 12 14 ( )an 310 510 710 Table shows only the order of the condition number and it is calculated here only with exact data. With more unfavourable dynamic properties of the system, ( )an grows only faster, which makes the problem of parametric identification with increasing an more sensitive to errors in the data, i.e. we have a manifestation of bad conditionali- ty of .AW This property is fundamental and does not essentially depend on the dynamic properties of systems and methods that solve the problem of parametric identification. Therefore, at larger dimensions, when the problem becomes ill-posed, it is necessary to use regularization procedures, which can be used to ensure the stability of the obtained solutions and improve the quality of the models. Poor conditionality is also inherent to stochastic identification problems, which led to a shift in a paradigm for system identi- fication [6]. The main attention in stochastic case was paid to the choice of various stabilizers that make it possible to ensure the consistency of the estimation. The most widely used is the Kernel structure for stabilizator, which contains parameters that are subject to tun- ing in order to provide optimal regularization. Various types of Kernel structure can be found in works [24–32]. Here in the framework of nonstohastic approach will be con- sidered regularization more familiar to classical one [33]. 5. Regularization Since an ill-posed problem arises when solving (24), it is proposed to find a solu- tion from (27) using the stabilizer stab in the following form: Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 51 1 2 stab 1 / 0 0 0 0 / 0 0 0 0 / 0 0 0 0 1 n n n n                         , where 1, 2 , …, an are singular values of 1. Instead of (27) we get the system 1 1 stab 1( ) ,y    (33) where  — is a regularization parameter, which is chosen from the residual principle by [33]. Parameter  changes in a geometric progression 0{ } { }, 0 1, 0,1, 2, ,k k k        where 0 is chosen to be of order 10-2. With 0k  we have an regularized solu- tion (0) a (0) 1 T 1 1 0 stab 1( ) .a V U y      (34) Without regularization, the vector (0) b  is found and the impulse response function of the model is calculated. It is compared with the impulse response function of the sys- tem, which can be taken as the free motion on the relaxation interval after excitation by a long rectangular pulse. Let 0ˆ ( )h t be the impulse response function of the model, and ( )h t — impulse response function of the systems. The accuracy of the solution is de- fined as 0 0ˆ ˆ( ) ( ) max ( ) ( ) . t h t h t h t h t     Next, we check the fulfillment of the condition 0ˆ( ) ( ) 2 .yh t h t     (35) When the condition (35) is met, we obtain a regularized solution that is consistent in accuracy with the data error. Otherwise, we take 2  and repeat the above steps. The decrease of  continues until (35) is fulfilled. If, starting from some ,k the monotonic character of the residual in (28) is violat- ed, we increase an and bn by one and continue the search for a regularized solution. The increase in dimensions is interrupted if there is no improvement in the quality of the model. In this case, the model of the smallest dimension is taken, among those that give a suitable approximation. 6. Simulation results This section presents experimental studies of the fundamental features of identifi- cation for systems with different dynamic properties and structures using numerical simulation. They are mainly determined by the system dimension and invariants, includ- ing eigenvalues and identifiability parameters characterizing the contribution of each individual mode to the output. 52 ISSN 2786-6491 To do this, we use the model representation (10), (11), in which the specified in- variants are represented by parameters ,p p and , .c s p pf f The complexity of the sys- tem is characterized by dimension .n Errors in measured output and input were as- sumed to be uniformly distributed over the membership intervals (7). For a given model dimension, the main characteristics that determine the quality of parameter estimation are the conditionality of the matrix AW in the overdetermined sys- tem (25) and the matrix BW in a similar system (31). The proposed excitation of the system by a large number of signals of types (12), (13) provides good conditionality of the matrices BW in a wide range of values for any system under study. The conditionality of the matrix AW essentially depends on the dynamic properties of the system, determined by its dimension and the values of the above invariants. As the dimension of the system and the approximating model increase, the conditionality grows rapidly. This is clearly seen from Fig. 1 and Fig. 2, where dependence of condi- tionality of the matrix AW on model order is shown for the systems of order 6 (Fig. 1) and of order 15 (Fig. 2). The dashed line shows the case with an uncertainty level of 610 . On Fig. 1, the data generating system has dimension 6,n  with two real and two complex conjugate eigenvalues. The conditionality of the system with 15,n  which has three real and six complex conjugate eigenvalues, is shown in Fig. 2. Fig. 1 Fig. 2 6 2 10 14 18 100 104 108 1012 1016 6 2 10 14 18 100 10 4 108 1012 1016 Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 53 The conditionality of the matrix AW for approximate models of different dimen- sions is shown to the left of .an n The solid line shows the deterministic case, when there is no uncertainty in the data. The dashed line shows the case with an uncertainty level of 610 . As can be seen from the figures, starting from a certain dimension, the condition number reached a constant level. In the deterministic case, the growth of conditionality stops when 1,an n  and is set at the level of the computational error 16(10 ). In the non-deterministic case, the increase in conditionality may stop earlier, as soon as it reaches a level corresponding to the uncertainty in the data 16(10 ). Then the dimen- sion is determined to be smaller than the true one, since some of the system modes have a signal with too low SNR. The results presented in Fig. 1, 2 are obtained for a well-identified system, when p is uniformly distributed over the interval 0, 2       , all p are close to unity, and all values of s pf and c pf are the same. If the eigenvalues are clustered or there are modes with relatively small coefficients s pf and ,c pf then the growth of the conditionality AW with increasing dimension accelerates even more. Fig. 3 Fig. 3 shows the conditionality of AW for a system with 15,n  whose eigenval- ues form clusters near the stability boundary. It can be seen that for a well-identified system, the condition number for model of order 10an  has a value of about 410 , and for a system with clustering of eigenvalues, it has a value of about 610 . The property of asymptotic convergence for the described identification method was investigated. The same system that was used to build Fig. 2 was chosen. The re- sponses of the system and asymptotic models for different an to a rectangular pulse are shown in Fig. 4–6. On Fig. 4 the model has a dimension of 10,an  in Fig. 5 — 13,an  in Fig. 6 — 15,an  i.e. coincided with the dimension of the original system. The plots to the left at the figures corresponds to the deterministic case and the right plots correspond to the non-deterministic case with input and output uncertainty of value 610 . True system response is shown as solid line, and model response is shown as dashed line. 6 2 10 14 18 100 104 108 1012 1016 54 ISSN 2786-6491 Fig. 4 Fig. 5 Fig. 6 Since the conditioning is bad in all cases, the regularization procedure (34), (35) was used. When approaching the exact dimension, the quality of the model improved. When an exceed ,n in the deterministic case, the coincidence of impulse response functions was obtained, which indicates the construction of a non-minimal model. For such models, the rank of the product of the controllability and observability matrices remained equal to .n In the non-deterministic case, the rank of the identifiability matrix increased with a probability of almost one, but the quality of the model did not improve significantly. At a higher level of uncertainty, this effect appears for asymptotic models starting from a certain dimension an less than ,n so further increase in dimension did not sig- nificantly improve the quality of the model. This value can be considered as the most appropriate dimension of the asymptotic model. It corresponds as a rule, to the stop of the growth of the conditionality of the matrix , anY according to Fig. 3. Thus, there is reason to believe that the dimension ,an at which the condition number reaches satura- tion, is the most appropriate for the approximating model. The dependence of the singular values of the matrices Y (15) and AW (25) on dy- namic invariants and the magnitude of the uncertainty was studied. It has been found that the singular values can be used to judge the dynamic properties of the system, the number of essentially excited modes, and the level of noise. 20 0 40 60 80 14 100 10 – 2 2 6 20 0 40 60 80 14 100 10 – 2 2 6 20 0 40 60 80 14 100 10 – 2 2 6 20 0 40 60 80 14 100 10 – 2 2 6 20 0 40 60 80 14 100 10 – 2 2 6 20 0 40 60 80 14 100 10 – 2 2 6 Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 55 For example, Fig. 7 shows the singular values of the matrix Y (15) with different noise levels. Generating system has dimension 15 with 6 complex conjugate pairs and 3 real eigenvalues. The values of the invariants c pf and s pf for different modes differ up to 100 times. Solid line shows singular values in the deterministic case, dashed and dashed-dotted lines correspond to the noise 610u y     and 310 ,u y     re- spectively. Fig. 7 Dealing specifically with this system in the deterministic case, one can hope to build a full-size model. In the case 610 ,u y     the successful construction of a full- size model is unlikely. In the case 310 ,u y     a model with 10an  is non- minimal, because all further modes will have an SNR below one. Conclusion The non-stochastic approach to the identification of complex systems in the framework of asymptotic modeling is more realistic for practical use, since it uses less stringent restrictions on the uncertainty in the data, including finite-time observation. To ensure the effectiveness of this method, two main requirements must be met. The first, it is necessary that the input signal be universally informative, i.e. for a system with arbitrary dynamic properties, the output of each mode of the system had an acceptable SNR. Otherwise, low SNR modes are poorly identified. The proposed design of active experiment with alternating intervals of excitation and relaxation partially solves this problem, due to the use of resonance effects. If the frequencies of the input signals vary with a small step, then the informativeness of the input becomes more uni- versal. In principle, the most versatile informative input signals are white noise signals. However, their implementation in practice can cause certain problems. The proposed method of excitation allows you to control its spectrum, which allows you to use, for example, a priori knowledge about the dynamic properties of the system. The second important aspect of non-stochastic identification is related to the choice of model dimension, which is the best in each particular case. To do this, within the framework of the described method, it is proposed to use the principle of choosing a model that is consistent in accuracy with the errors in the available data. In the first ap- proximation, it is used in condition (20). Further, for the dimension established by (23), the problem of parametric identification is solved, after which the consistency of the 6 2 10 14 18 10–15 10–9 10–3 100 56 ISSN 2786-6491 model and the system is checked by the difference in responses. If the principle is ful- filled, then the taken dimension of the model is final. If the model error is greater, we increase its dimension and perform parametric identification again. The principle of consistency is checked and a decision is made to stop or to continue increasing the di- mension of the model. In this case, if the problem is ill-posed, regularization should be used. However, the case is not ruled out when the quality of the model reaches satura- tion and does not improve. Then, as the desired dimension, we take the smallest one corresponding to saturation. In conclusion, we note that the process of identifying complex systems cannot be reduced to a formal solution of a strictly formulated mathematical problem. It is rather a research process of establishing patterns of system behavior based on the data of one or more experiments with the analysis of their results and the use of mathematical tools, the basics of which are described in this paper. В.Ф. Губарев, С.В. Мельничук, М.М. Сальніков ПАРАДИГМА НЕСТОХАСТИЧНОГО ПІДХОДУ ДО ІДЕНТИФІКАЦІЇ СИСТЕМ Губарев Вʼячеслав Федорович Інститут космічних досліджень НАН України та ДКА України, м. Київ, v.f.gubarev@gmail.com Мельничук Сергій Вікторович Інститут космічних досліджень НАН України та ДКА України, м. Київ, serg.vik@ukr.net Сальніков Микола Миколайович Інститут космічних досліджень НАН України та ДКА України, м. Київ, salnikov.nikolai@gmail.com Під поняттям складна система у цій роботі розуміється велика сукуп- ність динамічних взаємодіючих систем, точна математична модель яких невідома чи має дуже велику розмірність. Застосування стандарт- них методів синтезу зворотних звʼязків для таких систем стає складним і навіть неможливим із-за виродженості відповідних математичних за- дач. Один із виходів із такої ситуації полягає у побудові апроксимацій- ної моделі зниженої розмірності. Це може бути зроблено з використан- ням системи вихідних рівнянь, якщо вони є, або методів ідентифікації на основі вимірювань вихідних і вхідних змінних, що діють на систему. У цьому випадку процес побудови математичної моделі зводиться до послідовного перебору можливих моделей зі зростаючою складністю. Як критерій адекватності моделі розглядається норма відхилення вихо- ду моделі від виміряного значення виходу досліджуваної системи. У статті розглядається побудова лінійних моделей, складність яких ви- значається розмірністю. У рамках нестохастичного підходу розроблено методологічну та математичну основу реконструкції моделей, що опи- сують процеси у складних системах. Асимптотичне моделювання доз- воляє для такої системи формувати класи моделей, які підходять для розв’язання задачі ідентифікації. Точний опис відповідає нескінченно- му розширенню, тому якість моделі покращується зі збільшенням її роз- мірності. Однак помилки в наявних даних не дозволяють безмежно mailto:v.f.gubarev@gmail.com mailto:serg.vik@ukr.net mailto:salnikov.nikolai@gmail.com Міжнародний науково-технічний журнал Проблеми керування та інформатики, 2023, № 1 57 збільшувати їхню розмірність через погану обумовленість задачі іден- тифікації, починаючи з деякого виміру. Процедура регуляризації до- зволяє визначити ефективне наближене рішення задачі ідентифікації, яке для нестохастичного випадку узгоджується з помилками даних. Властивості та особливості пропонованого підходу ілюструються ре- зультатами моделювання. Kлючові слова: ідентифікація системи, лінійна регресія, регуляризація, асимптотичне моделювання, наближений розв’язок. REFERENCES 1. Glover K., Curtain R.F., Partington J.R. Realization and approximation of linear infinite-dimen- sional systems with error bounds. SIAM Journal of Control and Optimization. 1988. Vol. 26, N 4. P. 863–898. https://doi.org/10.1137/0326049. 2. Ljung L. System identification — theory for the user (2nd ed.). Upper Saddle River, NJ : Pren- tice-Hall, 1999. 631 p. 3. Wahlberg B. On the use of orthogonalized exponentials in system identification. In Report LiTH- ISY-1099, Department of Electrical Engeneering. Linkoping University, Sweden, 1990. 137 p. Wahlberg B. System identification using Laguerre models. IEEE Transactions on Automatic Con- trol. 1991. Vol. 36, N 5. P. 551–562. DOI:10.1109/9.76361 4. Van den Hof P.M.J., Heuberger P.S.C., Bokor J. System identification with generalized or- thonormal basis functions. Automatica. 1995. Vol. 32, N 12. P. 1821–1834. DOI:10.1016/0005- 1098(95)00074-4 5. Ljung L., Chen T., Mu B. A shift in paradigm for system identification. International Journal of Control. 2020. Vol. 93, N 2. P. 173–180. DOI:10.1080/00207179.2019.1578407 6. Söderström T., Stoica P. System identification. U.K. : Prentice-Hall, 1989. 637 p. 7. Pintelon R., Schoukens J. System identification — a frequency domain approach. New York, NY : IEEE Pres., 2001. 648 p. 8. Söderström T. Accuracy analysis of the Frisch scheme for identifying errors-in-variables system. IEEE Transaction on Automatic Control. 2007. Vol. 52, N 6. P. 985–997. doi: 10.1109/TAC.- 2007.899005. 9. Söderström T. Errors-in-variables methods in system identification. Automatica. 2007. Vol. 43, N 6. P. 939–958. DOI:10.1016/j.automatica.2006.11.025 10. Van Huffel S., Lemmerling P.(Eds.). Total least squares and errors-invariables modelling: Analy- sis, algorithms and applications. Springer, 2002. 388 p. 11. Guidorzi R.P., Beghlli S., Soverini U. The frisch scheme in dynamic system identification. Auto- matica. 1990. Vol. 26, N 1. P. 171–176. DOI:10.1016/0005-1098(90)90168-H 12. Hong M., Söderström T., Zheng W.X. Accuracy analysis of bias-eliminating least squares esti- mates for errors-in-variables systems. Automatica. 2007. Vol. 43, N 9. P. 1590–1596. DOI:10.1016/j.automatica.2007.02.002 13. Söderström T., Soverini U. Errors-in-variables identification using maximum likelihood estima- tion in the frequency domain. Automatica. 2017. Vol. 79. P. 131–143. DOI:10.1016/- j.automatica.2017.01.016 14. Zhang E., Pintelon R. Identification of multivariable dynamic errors-in-variables system with ar- bitrary inputs. Automatica. 2017. Vol. 82. P. 69–78. DOI:10.1016/j.automatica.2017.04.031 15. Zhang E., Pintelon R. Nonparametric identification of linear dynamic errors-in-variables systems. Automatica. 2018. Vol. 94. P. 416–425. DOI:10.1016/j.automatica.2018.04.039 16. Kang H., Gu G., Zheng W.X. A graph subspace approach to system identification based on er- rors-in-variables system models. Automatica. 2019. Vol. 109. DOI:10.1016/j.automatica.- 2019.108535 17. Milanese M., Norton J., Piet-Lahanier H., Walter, E.(eds). Bounding Approaches to System Iden- tification. New York : Plenum Press, 1996. 586 p. 18. Garulli A., Reinelt W. On model error modeling in set membership identification. IFAC Proceedings Volumes. 2000. Vol. 33, N 15. P. 169–174. DOI:10.1016/S1474-6670(17)- 39745-8. 19. Giarre L., Kacewicz B.Z., Milanese M. Model quality evaluation in set membership identifica- tion. Automatica. 1997. Vol. 33, N 6. P. 1133–1139. DOI:10.1016/S0005-1098(97)00007-1. 20. Milanese M., Novara C. Set membership identification of nonlinear systems. Automatica. 2004. Vol. 40, N 6. P. 957–975. DOI:10.1016/j.automatica.2004.02.002 https://doi.org/10.1137/0326049 https://doi.org/10.1109/9.76361 https://doi.org/%1f10.1016/0005-1098(95)00074-4 https://doi.org/%1f10.1016/0005-1098(95)00074-4 https://doi.org/10.1080/00207179.2019.1578407 https://doi.org/10.1016/j.automatica.2006.11.025 https://doi.org/10.1016/0005-1098(90)90168-H https://doi.org/10.1016/j.automatica.2007.02.002 https://doi.org/10.1016/j.automatica.2017.01.016 https://doi.org/10.1016/j.automatica.2017.01.016 https://doi.org/10.1016/j.automatica.2017.04.031 https://doi.org/10.1016/j.automatica.2018.04.039 https://doi.org/10.1016/j.%1fautomatica.2019.108535 https://doi.org/10.1016/j.%1fautomatica.2019.108535 https://doi.org/10.1016/S0005-1098(97)%1f00007-1 https://doi.org/10.1016/j.automatica.2004.02.002 58 ISSN 2786-6491 21. Gubarev V., Melnychuk S., Salnikov N. Identification of complex systems in the class of linear regression models. In Recent Developments in Automatic Control Systems. Edited by: Kondratenko Y.P., Kuntsevich V.M., Chikrii A.A., Gubarev V.F. River Publishers, 2022. P. 167–195. 22. Golub G.H., Van Loan C.F. Matrix computations. London : Johns Hopkins University Press, 2012. 694 p. 23. Pillonetto G., Chiuso A., De Nicolao G. Regularized estimation of sums of exponentials in spaces generated by stable spline kernels. Proc. 2010 American Control Conference (Baltimor, MD: Marriott Waterfront). P. 498–503. DOI:10.1109/ACC.2010.5530862 24. Pillonetto G., Chiuso A., De Nicolao G. Prediction error identification of linear systems: A nonpar- ametric Gaussian regression approach identification. Automatica. 2010. Vol. 47, N 2. P. 291–305. DOI:10.1016/j.automatica.2010.11.004 25. Chen T., Andersen M.S., Ljung L., Chiuso A., Pillonetto G. System identification via sparse mul- tiple kernel-based regularization using sequential convex optimization techniques. IEEE Transac- tions on Automatic Control. 2014. Vol. 59, N 11. P. 2933–2945. DOI:10.1109/TAC.2014.- 2351851. 26. Chen T., Ohlsson H., Ljung L. On the estimation of transfer functions, regularizations and Gauss- ian processes — revisited. Automatica. 2012. Vol. 48, N 8. P. 1525–1535. DOI:10.1016/- j.automatica.2012.05.026 27. Chen T., Ardeshiri T., Carli F.P., Chiuso A., Ljung L., Pillonetto G. Maximum entropy properties of discrete-time first-order stable spline kernel. Automatica. 2015. Vol. 66. P. 34–38. DOI:10.- 1016/j.automatica.2015.12.009 28. Dinuzzo F. Kernel for linear time invariant system identification. SIAM Journal on Control and Optimization. 2015. Vol. 53, N 5. P. 3299–3317. DOI:10.1137/130920319 29. Carli F.P., Chen T., Ljung L. Maximum entropy kernels for system identification. IEEE Transactions on Automatic Control. 2017. Vol. 63, N 3. P. 1471–1477. DOI:10.1109/- TAC.2016.2582642. 30. Markonato A., Schoukens M., Schoukens J. Filter-based regularization for impulse response modelling. IET Control Theory & Applications. 2017. Vol. 11, N 2. P. 194–204. DOI:10.1049/iet- cta.2016.0908 31. Zorzi M., Chiuso A. The harmonic analysis of kernel functions. Automatica. 2018. Vol. 94. P. 125–137. DOI:10.1016/j.automatica.2018.04.015 32. Tikhonov A.N., Arsenin V.Y. Solution of ill-posed problems. Washington, DC : Winston/Wiley, 1977. 272 p. Submitted 14.03.2023 https://doi.org/10.1109/ACC.2010.5530862 https://doi.org/10.1137/130920319 https://doi.org/10.1016/j.automatica.2018.04.015
id nasplib_isofts_kiev_ua-123456789-210935
institution Digital Library of Periodicals of National Academy of Sciences of Ukraine
issn 0572-2691
language Ukrainian
last_indexed 2026-03-13T00:52:50Z
publishDate 2023
publisher Інститут кібернетики ім. В.М. Глушкова НАН України
record_format dspace
spelling Gubarev, V.
Melnychuk, S.
Salnikov, N.
2025-12-21T10:53:01Z
2023
Paradigm of nonstochastic approach tosystem identification / V. Gubarev, S. Melnychuk, N. Salnikov // Проблеми керування та інформатики. — 2023. — № 1. — С. 42–58. — Бібліогр.: 32 назв. — англ.
0572-2691
https://nasplib.isofts.kiev.ua/handle/123456789/210935
681.5.015; 519.233
10.34229/1028-0979-2023-1-4
The paper discusses the construction of linear models, the complexity of which is determined by dimensionality. Within the framework of the non-stochastic approach, a methodological and mathematical foundation for reconstructing models that describe processes in complex systems is developed. Asymptotic modeling allows for the formation of classes of models suitable for solving identification problems. An exact description corresponds to an infinite extension, and thus the quality of the model improves with its increasing dimensionality. However, errors in the available data prevent the indefinite increase in dimensionality due to the ill-posedness of the identification problem starting from a certain dimension. The regularization procedure enables the determination of an effective approximate solution to the identification problem, which, in the non-stochastic case, is consistent with the data errors. The properties and features of the proposed approach are illustrated by modeling results.
У статті розглядається побудова лінійних моделей, складність яких визначається розмірністю. У рамках нестохастичного підходу розроблено методологічну та математичну основу реконструкції моделей, що описують процеси у складних системах. Асимптотичне моделювання доз­воляє для такої системи формувати класи моделей, які підходять для розв’язання задачі ідентифікації. Точний опис відповідає нескінченному розширенню, тому якість моделі покращується зі збільшенням її роз­мірності. Однак помилки в наявних даних не дозволяють безмежно збільшувати їхню розмірність через погану обумовленість задачі ідентифікації, починаючи з деякого виміру. Процедура регуляризації дозволяє визначити ефективне наближене рішення задачі ідентифікації, яке для нестохастичного випадку узгоджується з помилками даних. Властивості та особливості пропонованого підходу ілюструються результатами моделювання.
uk
Інститут кібернетики ім. В.М. Глушкова НАН України
Проблеми керування та інформатики
Методи керування та оцінювання в умовах невизначеності
Paradigm of nonstochastic approach tosystem identification
Парадигма нестохастичного підходу до ідентифікації систем
Article
published earlier
spellingShingle Paradigm of nonstochastic approach tosystem identification
Gubarev, V.
Melnychuk, S.
Salnikov, N.
Методи керування та оцінювання в умовах невизначеності
title Paradigm of nonstochastic approach tosystem identification
title_alt Парадигма нестохастичного підходу до ідентифікації систем
title_full Paradigm of nonstochastic approach tosystem identification
title_fullStr Paradigm of nonstochastic approach tosystem identification
title_full_unstemmed Paradigm of nonstochastic approach tosystem identification
title_short Paradigm of nonstochastic approach tosystem identification
title_sort paradigm of nonstochastic approach tosystem identification
topic Методи керування та оцінювання в умовах невизначеності
topic_facet Методи керування та оцінювання в умовах невизначеності
url https://nasplib.isofts.kiev.ua/handle/123456789/210935
work_keys_str_mv AT gubarevv paradigmofnonstochasticapproachtosystemidentification
AT melnychuks paradigmofnonstochasticapproachtosystemidentification
AT salnikovn paradigmofnonstochasticapproachtosystemidentification
AT gubarevv paradigmanestohastičnogopídhodudoídentifíkacíísistem
AT melnychuks paradigmanestohastičnogopídhodudoídentifíkacíísistem
AT salnikovn paradigmanestohastičnogopídhodudoídentifíkacíísistem