Parallel algorithm for the identification of distributed scatterers in the problem of calculating the velocities of displacements of the earth’s surface by the Persistent Scaterrers method

Описывается программная реализация быстрого алгоритма поиска распределенных рассеивателей для задачи построения скоростей смещений земной поверхности на базе платформы Apache Spark. Рассматривается полная схема расчета скоростей смещений методом постоянных рассеивателей. Предложенный алгоритм интегрируется в схему после этапа совмещения с субпиксельной точностью стека изображений временн´ой серии радарных снимков космического аппарата Sentinel-1. Алгоритм не является итерационным и может быть реализован в парадигме параллельных вычислений. Применяемая платформа Apache Spark позволила распределенно обрабатывать массивы стека радарных данных (от 60 изображений) в памяти на большом количестве физических узлов в сетевой среде. Время поиска распределенных рассеивателей удалось снизить в среднем до десяти раз по сравнению с однопроцессорной реализацией алгоритма. Приведены сравнительные результаты тестирования вычислительной системы на демонстрационном кластере. Алгоритм реализован на языке программирования Python c подробным описанием методов и объектов The article describes implementation of the software for a fast algorithm which finds distributed scatterers for the problem of plotting displacement velocities of the earth’s surface based on the Apache Spark platform. The Persistent Scatterer (PS) method is widely used for estimating the displacement rates of the earth’s surface. It consists of the identification of coherent radar targets (interferogram pixels) that demonstrate high phase stability during the entire observation period. The most advanced algorithm for solving the identification problem is the SqueeSAR algorithm. It allows searching and processing Distributed Scatterers (DS) - specific reflectors, integrating them into the general scheme for calculating displacement velocities using the PS method. A careful analysis of the SqueeSAR algorithm has identified areas that are critical to its performance. The whole algorithm is based on an enumeration of the initial data, where nontrivial transformations are performed at each step. The stages of searching for adjacent points in the design window with multiple passes over the entire area of the image and solving the maximization problem when assessing the real values of the interferometric phases turned out to be noticeably costly. To speed up the processing of images, it is proposed to use the Apache Spark massively parallel computing platform. Specialized primitives (Resilient Distributed Data) for recurrent inmemory processing are available here. This provides multiple accesses to the radar data loaded into memory from each cluster node and allows logical dividing of the snapshot stack into subareas. Thus calculations are performed independently in massively parallel mode. Based on the SqueeSAR mathematical model, it is assumed that the radar image data and the calculated geophysical parameters calculated are common for each statistically homogeneous sample of nearby pixels. In accordance with this assumption, the uniformity (homogeneity) of the pixels is estimated within a given window. The search for distributed scatterers occurs independently by the sequence of shifts of the windows over the entire area of the image. The window is shifted along the width and height of the image with a step equal to the width and height of the window. Pairs of samples in the window are composed of vectors of complex pixel values in each of the N images. The validity of the Kolmogorov-Smirnov criterion is checked for each of the pairs. To estimate the values of the phases of homogeneous pixels, the maximization problem is solved. The method of maximum likelihood estimation (MLE) is considered. The construction of the correct MLE form is carried out by analyzing the statistical properties of the coherence matrix of all images using the complex Wishart distribution. The Apache Spark platform applied here permits processing of distributed radar data stack arrays in memory on a large number of physical nodes in a network environment. The average search time for distributed scatterers turned out to be 10 times less compared to the uniprocessor implementation of the algorithm. The algorithm is implemented in the Python programming language with a detailed description of the objects and methods of the algorithm. The proposed algorithm and its parallel implementation allows applying the developed approaches to other problems and types of satellite data for remote sensing of the earth from space