A generalized methodology of sign language recognition on video streams based on neural networks and transformers

• Дата: 2023
• Автори: Кузнєцова, Н. В., Смірнов, С. С.
• Формат: Стаття
• Мова: Ukrainian
• Опубліковано: Інститут проблем реєстрації інформації НАН України 2023
• Теми: системна методологія; системний аналіз; нейронні мережі; трансформери; мова жестів; розпізнавання; рекурентні нейронні мережі; згорткові нейронні мережі; system methodology; system analysis; neural networks; transformers; sign language; recognition; recurrent neural networks; convolutional neural networks
• Онлайн доступ: http://drsp.ipri.kiev.ua/article/view/300527
• Назва журналу: Data Recording, Storage & Processing

The investigation presents a systematic methodology for effective sign language recognition on video streams which considers the importance of using neural networks and transformers for automatic detection and gestures recognition in real time. This methodology combines such fields as computer vision and natural language processing and also uses transformers as artificial intelligence models in aim to effectively model long-term dependencies in data sequences. The study describes the main stages of the methodology development, correct problem statement definition and the main descriptions of system methodology. A review of existing approaches and methodologies for the task solving was also carried out by authors focusing on increasing the accuracy and speed of video streams processing, as well as the task of conducting the recognition process in real time. The work provides an analysis of the proposed methodology stability with respect to problems and challenges that may arise during implementation and application to real data. The meaning of the proposed methodology lies in its unique approach which simultaneously works both in the domain of computer vision and in natural language processing. The use of transformers allows one to effectively take into account complex structures and dependencies between gestures which lead to increasing the accuracy and speed recognition. This study is an important contribution to the development of sign language recognition systems and is noted for its innovative potential for further development, validation and as next practical usage in various fields, including human-machine interfaces, security systems and virtual reality. Fig.: 4. Refs: 14 titles.