Search

The significance of earth observation data spans diverse fields and domains, driving the need for efficient management. Nevertheless, the exponential increase in data volume brings new challenges that complicate processing and storing data. This article proposes an optimized multi-modular service for earth observation data management in response to these challenges. The suggested approach focuses on choosing the optimal configurations for the storage and processing layers to improve the performance and cost-effectiveness of managing data. By employing the recommended optimized strategies, earth observation data can be managed more effectively, resulting in fast data processing and reduced costs. [ABSTRACT FROM AUTHOR]

Earth observation technology has become increasingly crucial for monitoring various aspects of our planet through systematic data collection. However, processing the large volume of satellite data generated can be challenging, requiring high-performance computing solutions. The Dask framework has gained popularity for its flexibility and efficiency in processing large amounts of data in a distributed manner. Nevertheless, determining the optimal Dask cluster configuration remains challenging, as it requires balancing performance and cost objectives. A novel multi-objective optimization service is proposed to address this challenge that enhances the performance and cost efficiency of earth observation data processing workflows. Our approach is to generate a set of Pareto-optimal solutions, allowing users to make informed decisions regarding the optimal trade-offs between performance and cost. The effectiveness is demonstrated using real-world earth observation datasets and outperforms existing performance and cost-efficiency solutions. [ABSTRACT FROM AUTHOR]