Проблема автоматичної класифікації зображень за використання інтелектуальної системи прийняття рішень на основі графа знань і точного аналізу зображень

• Дата: 2022
• Автори: Martynenko, Andrii, Tevyashev, Andriy, Kulishova, Nonna, Moroz, Boris
• Формат: Стаття
• Мова: English
• Опубліковано: The National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute" 2022
• Теми: автоматична багатозадачна класифікація; граф знань; механізм уваги; дрібнодетальний аналіз зображень; музейна експертиза; твори живопису; згорткові нейронні мережі; automatic multi-task classification; knowledge graph; attention mechanism; fine-grained image analysis; museum expertise; paintings; convolutional neural networks
• Онлайн доступ: http://journal.iasa.kpi.ua/article/view/275082
• Назва журналу: System research and information technologies

In order to prevent the illegal export of paintings abroad, a museum examination using various methods for studying a work of art is carried out. At the same time, an analysis is also made of historical, art history, financial and other information and documents confirming the painting’s authenticity — provenance. Automation of such examination is hampered by the need to take into account numerical values of visual features, quality indicators, and verbal descriptions from provenance. In this paper, we consider the problem of automatic multi-task classification of paintings for museum expertise. A system architecture is proposed that checks provenance, implements a fine-grained image analysis (FGIA) of visual image features, and automatically classifies a painting by authorship, genre, and time of creation. Provenance is contained in a knowledge graph; for its vectorization, it is proposed to use a graph2vec type encoder with an attention mechanism. Fine-grained image analysis is proposed to be performed using searching discriminative regions (SDR) and learning discriminative regions (LDR) allocated by convolutional neural networks. To train the classifier, a generalized loss function is proposed. A data set is also proposed, including provenance and images of paintings by European and Ukrainian artists.