Method of the coherence evaluation of Ukrainian text.

• Дата: 2019
• Автори: Pogorilyy, S. D., Kramov, A. A.
• Формат: Стаття
• Мова: Ukrainian
• Опубліковано: Інститут проблем реєстрації інформації НАН України 2019
• Теми: обробка природної мови; когерентність тексту; граф семантичної схожості; нейронна мережа; модель Doc2Vec; семантична міра схожості речень; natural language processing; text coherence; semantic similarity graph; neural network; Doc2Vec model; semantic similarity value of sentences
• Онлайн доступ: http://drsp.ipri.kiev.ua/article/view/178945
• Назва журналу: Data Recording, Storage & Processing

Due to the growing role of the SEO technologies, it is necessary to perform an automated analysis of the article’s quality. Such an approach helps both to return the most intelligible pages for the user’s query and to raise the web-sites positions to the top of query results. An automated assessment of a cohe-rence is a part of the complex analysis of the text. In this article, main methods for text coherence measurements for Ukrainian languageare analyzed. Expediency of using the semantic similarity graph method in comparison with other methods is explained. It is suggested the improvement of that method by the pre-training of the neural network for vector representations of sentences. Experimental examination of the original method and its modifications is made. Training and examination procedures are made on the corpus of Ukrainian texts, which were previously retrieved from abstracts and full texts of Ukrainian scientific articles. The testing procedure is implemented by performing of two typical tasks for the text coherence assessment: document discrimination task and insertion task. Accordingly to the analysis it is defined the most effective combination of method’s modification and its parameter for the measurement of the text coherence.