Software system for analyzing cloudiness based on satellite observations

• Дата: 2019
• Автори: Doroshenko, А.Yu., Shpyg, V.M., Ovdii, O.M.
• Формат: Стаття
• Мова: Ukrainian
• Опубліковано: PROBLEMS IN PROGRAMMING 2019
• Теми: distributed computing; workflow; Apache Hadoop; Apache Spark; image analysis; monitoring; cloudiness; UDC 004.932.2:[551.501:551.576.2]
• Онлайн доступ: https://pp.isofts.kiev.ua/index.php/ojs1/article/view/370
• Назва журналу: Problems in programming

The article presents software system for analyzing cloudiness indicators in Ukraine based on satellite observations. This system was developed using an online dialog designer for automated workflow design based on algebra-algorithmic and ontological tools ODWF. The workflow for the implementation of the task was designed for Apache Oozie and distributed platform Apache Hadoop. In order to support scalability and optimize the processing of large volumes of satellite images, the Apache Spark framework was used to implement image analysis. An experiment was conducted to run the developed workflow on a different number of Apache Spark threads, the results of which demonstrated a good efficiency of calculation parallelization. It is shown that the results obtained during the experiment at a qualitative level are in good agreement with the data of long-term ground-based observations and actual observational data.Problems in programming 2019; 3: 116-126