A method for improving the expert estimates consistency in the dialogue process
Improving the quality of recommendations for decision-makers in weakly structured domains is affected largely by the reliability of results of appraisals carried out during the construction of models of weakly structured systems based on which recommendations for decision-makers are generated. The r...
Збережено в:
| Дата: | 2018 |
|---|---|
| Автори: | , |
| Формат: | Стаття |
| Мова: | Ukrainian |
| Опубліковано: |
Інститут проблем реєстрації інформації НАН України
2018
|
| Теми: | |
| Онлайн доступ: | http://drsp.ipri.kiev.ua/article/view/142915 |
| Теги: |
Додати тег
Немає тегів, Будьте першим, хто поставить тег для цього запису!
|
| Назва журналу: | Data Recording, Storage & Processing |
Репозитарії
Data Recording, Storage & Processing| id |
drspiprikievua-article-142915 |
|---|---|
| record_format |
ojs |
| institution |
Data Recording, Storage & Processing |
| baseUrl_str |
|
| datestamp_date |
2019-12-27T07:17:30Z |
| collection |
OJS |
| language |
Ukrainian |
| topic |
decision-making support expert estimations of expert estimates sufficient level of consistency for aggregation feedback with experts |
| spellingShingle |
decision-making support expert estimations of expert estimates sufficient level of consistency for aggregation feedback with experts Roik, P. D. Tsyganok, V. V. A method for improving the expert estimates consistency in the dialogue process |
| topic_facet |
decision-making support expert estimations of expert estimates sufficient level of consistency for aggregation feedback with experts поддержка принятия решений экспертные оценки согласованность экспертных оценок достаточный для агрегации уровень согласованности обратная связь с экспертами підтримка прийняття рішень експертні оцінки узгодженість експертних оцінок достатній для агрегації рівень узгодженості зворотний зв’язок з експертами |
| format |
Article |
| author |
Roik, P. D. Tsyganok, V. V. |
| author_facet |
Roik, P. D. Tsyganok, V. V. |
| author_sort |
Roik, P. D. |
| title |
A method for improving the expert estimates consistency in the dialogue process |
| title_short |
A method for improving the expert estimates consistency in the dialogue process |
| title_full |
A method for improving the expert estimates consistency in the dialogue process |
| title_fullStr |
A method for improving the expert estimates consistency in the dialogue process |
| title_full_unstemmed |
A method for improving the expert estimates consistency in the dialogue process |
| title_sort |
method for improving the expert estimates consistency in the dialogue process |
| title_alt |
Метод улучшения согласованности оценок экспертов в процессе диалога Метод поліпшення узгодженості оцінок експертів у ході діалогу |
| description |
Improving the quality of recommendations for decision-makers in weakly structured domains is affected largely by the reliability of results of appraisals carried out during the construction of models of weakly structured systems based on which recommendations for decision-makers are generated. The result of every group expert appraisal is a rating obtained through the aggregation of individual expert estimates. Usually, aggregation is justified when the consistency rate achieves a certain level (threshold of consistency), otherwise the result would be unreliable. It is offered a non-statistical approach to the evaluation of consistency, using which an expert estimates are compared in pairs (simplest case). A rate of consistency between the expert estimates sufficient for their aggregation has been identified. The evaluation of this threshold value is based on a statistical approach, triangular distribution and distribution function. A method has been developed for improving the consistency between expert estimates: it is based on a spectral approach where a set of estimates is presented as spectrum elements on a discrete or a continuous positive scale. The method offered for dialogue with experts minimizes tensions put on them, and thus allows raising the consistency between their estimates to a sufficient level and increases significantly the probability of successful completion of appraisal. The technique offered allows optimizing the task of raising the rate of consistency both thanks to using more reliable algorithms for evaluating such consistency and thanks to the developed step-by-step procedure of improvements through questions asked to experts, allowing at the same time to minimize the impact of the subjectivity of their estimates on the reliability of research. The method uses optimization allowing for automatic identification of the shortest path for improving consistency. This allows addressing each expert only once, without putting tensions on him/her by indicating the desired direction in changing his/her estimate. The methods and procedures offered have been implemented in a web-distributed system of support for decision-making and have been approved in practical appraisals. |
| publisher |
Інститут проблем реєстрації інформації НАН України |
| publishDate |
2018 |
| url |
http://drsp.ipri.kiev.ua/article/view/142915 |
| work_keys_str_mv |
AT roikpd amethodforimprovingtheexpertestimatesconsistencyinthedialogueprocess AT tsyganokvv amethodforimprovingtheexpertestimatesconsistencyinthedialogueprocess AT roikpd metodulučšeniâsoglasovannostiocenokékspertovvprocessedialoga AT tsyganokvv metodulučšeniâsoglasovannostiocenokékspertovvprocessedialoga AT roikpd metodpolípšennâuzgodženostíocínokekspertívuhodídíalogu AT tsyganokvv metodpolípšennâuzgodženostíocínokekspertívuhodídíalogu AT roikpd methodforimprovingtheexpertestimatesconsistencyinthedialogueprocess AT tsyganokvv methodforimprovingtheexpertestimatesconsistencyinthedialogueprocess |
| first_indexed |
2025-07-17T10:57:07Z |
| last_indexed |
2025-07-17T10:57:07Z |
| _version_ |
1850411255831986176 |
| spelling |
drspiprikievua-article-1429152019-12-27T07:17:30Z A method for improving the expert estimates consistency in the dialogue process Метод улучшения согласованности оценок экспертов в процессе диалога Метод поліпшення узгодженості оцінок експертів у ході діалогу Roik, P. D. Tsyganok, V. V. decision-making support expert estimations of expert estimates sufficient level of consistency for aggregation feedback with experts поддержка принятия решений экспертные оценки согласованность экспертных оценок достаточный для агрегации уровень согласованности обратная связь с экспертами підтримка прийняття рішень експертні оцінки узгодженість експертних оцінок достатній для агрегації рівень узгодженості зворотний зв’язок з експертами Improving the quality of recommendations for decision-makers in weakly structured domains is affected largely by the reliability of results of appraisals carried out during the construction of models of weakly structured systems based on which recommendations for decision-makers are generated. The result of every group expert appraisal is a rating obtained through the aggregation of individual expert estimates. Usually, aggregation is justified when the consistency rate achieves a certain level (threshold of consistency), otherwise the result would be unreliable. It is offered a non-statistical approach to the evaluation of consistency, using which an expert estimates are compared in pairs (simplest case). A rate of consistency between the expert estimates sufficient for their aggregation has been identified. The evaluation of this threshold value is based on a statistical approach, triangular distribution and distribution function. A method has been developed for improving the consistency between expert estimates: it is based on a spectral approach where a set of estimates is presented as spectrum elements on a discrete or a continuous positive scale. The method offered for dialogue with experts minimizes tensions put on them, and thus allows raising the consistency between their estimates to a sufficient level and increases significantly the probability of successful completion of appraisal. The technique offered allows optimizing the task of raising the rate of consistency both thanks to using more reliable algorithms for evaluating such consistency and thanks to the developed step-by-step procedure of improvements through questions asked to experts, allowing at the same time to minimize the impact of the subjectivity of their estimates on the reliability of research. The method uses optimization allowing for automatic identification of the shortest path for improving consistency. This allows addressing each expert only once, without putting tensions on him/her by indicating the desired direction in changing his/her estimate. The methods and procedures offered have been implemented in a web-distributed system of support for decision-making and have been approved in practical appraisals. Разработан метод улучшения согласованности экспертных оценок, которая определяется на основе спектрального подхода, когда множество оценок подается в виде составляющих спектра на дискретной или непрерывной положительной шкале. Определен уровень (порог) согласованности экспертных оценок, достаточный для их агрегации. Предложенный метод в диалоге с экспертами, при сведении к минимуму давления на них, позволяет повысить согласованность их оценок до достаточного уровня и значительно повышает вероятность успешного завершения экспертиз. Розроблено метод поліпшення узгодженості експертних оцінок, яка визначається на основі спектрального підходу, коли множина оцінок подається у вигляді складових спектра на дискретній або неперервній додатній шкалі. Визначено рівень (поріг) узгодженості експертних оцінок, що є достатнім для їхньої агрегації. Запропонований метод у діалозі з експертами, при зведенні до мінімуму тиску на них, дозволяє підвищити узгодженість їхніх оцінок до достатнього рівня та значно підвищує ймовірність успішного завершення експертиз. Інститут проблем реєстрації інформації НАН України 2018-06-19 Article Article application/pdf http://drsp.ipri.kiev.ua/article/view/142915 10.35681/1560-9189.2018.20.2.142915 Data Recording, Storage & Processing; Vol. 20 No. 2 (2018); 85–95 Регистрация, хранение и обработка данных; Том 20 № 2 (2018); 85–95 Реєстрація, зберігання і обробка даних; Том 20 № 2 (2018); 85–95 1560-9189 uk http://drsp.ipri.kiev.ua/article/view/142915/140362 Авторське право (c) 2021 Реєстрація, зберігання і обробка даних |