Search

The synergy between artificial intelligence (AI) and hyperspectral imaging (HSI) holds tremendous potential across a wide array of fields. By leveraging AI, the processing and interpretation of the vast and complex data generated by HSI are significantly enhanced, allowing for more accurate, efficient, and insightful analysis. This powerful combination has the potential to revolutionize key areas such as agriculture, environmental monitoring, and medical diagnostics by providing precise, real-time insights that were previously unattainable. In agriculture, for instance, AI-driven HSI can enable more precise crop monitoring and disease detection, optimizing yields and reducing waste. In environmental monitoring, this technology can track changes in ecosystems with unprecedented detail, aiding in conservation efforts and disaster response. In medical diagnostics, AI-HSI could enable earlier and more accurate disease detection, improving patient outcomes. As AI algorithms advance, their integration with HSI is expected to drive innovations and enhance decision-making across various sectors. The continued development of these technologies is likely to open new frontiers in scientific research and practical applications, providing more powerful and accessible tools for a wider range of users. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) significantly enhances the development of Meta-Optics (MOs), which encompasses advanced optical components like metalenses and metasurfaces designed to manipulate light at the nanoscale. The intricate design of these components requires sophisticated modeling and optimization to achieve precise control over light behavior, tasks for which AI is exceptionally well-suited. Machine learning (ML) algorithms can analyze extensive datasets and simulate numerous design variations to identify the most effective configurations, drastically speeding up the development process. AI also enables adaptive MOs that can dynamically adjust to changing imaging conditions, improving performance in real-time. This results in superior image quality, higher resolution, and new functionalities across various applications, including microscopy, medical diagnostics, and consumer electronics. The combination of AI with MOs thus epitomizes a transformative advancement, pushing the boundaries of what is possible in imaging technology. In this review, we explored the latest advancements in AI-powered metalenses for imaging applications. [ABSTRACT FROM AUTHOR]

Currently, we can solve a wide range of tasks using computer vision algorithms, which reduce manual labor and enable rapid analysis of the environment. The remote sensing domain provides vast amounts of satellite data, but it also poses challenges associated with processing this data. Baseline solutions with intermediate results are available for various tasks, such as forest species classification, infrastructure recognition, and emergency situation analysis using satellite data. Despite these advances, two major issues with high-performing artificial intelligence algorithms remain in the current decade. The first issue relates to the availability of data. To train a robust algorithm, a reasonable amount of well-annotated training data is required. The second issue is the availability of satellite data, which is another concern. Even though there are a number of data providers, high-resolution and up-to-date imagery is extremely expensive. This paper aims to address these challenges by proposing an effective pipeline for building segmentation that utilizes freely available Sentinel-2 data with 10 m spatial resolution. The approach we use combines a super-resolution (SR) component with a semantic segmentation component. As a result, we simultaneously consider and analyze SR and building segmentation tasks to improve the quality of the infrastructure analysis through medium-resolution satellite data. Additionally, we collected and made available a unique dataset for the Russian Federation covering area of 1091.2 square kilometers. The dataset provides Sentinel-2 imagery adjusted to the spatial resolution of 2.5 m and is accompanied by semantic segmentation masks. The building footprints were created using OpenStreetMap data that was manually checked and verified. Several experiments were conducted for the SR task, using advanced image SR methods such as the diffusion-based SR3 model, RCAN, SRGAN, and MCGR. The MCGR network produced the best result, with a PSNR of 27.54 and SSIM of 0.79. The obtained SR images were then used to tackle the building segmentation task with different neural network models, including DeepLabV3 with different encoders, SWIN, and Twins transformers. The SWIN transformer achieved the best results, with an F1-score of 79.60. [ABSTRACT FROM AUTHOR]

Large datasets catalyze the rapid expansion of deep learning and computer vision. At the same time, in many domains, there is a lack of training data, which may become an obstacle for the practical application of deep computer vision models. To overcome this problem, it is popular to apply image augmentation. When a dataset contains instance segmentation masks, it is possible to apply instance-level augmentation. It operates by cutting an instance from the original image and pasting to new backgrounds. This article challenges a dataset with the same objects present in various domains. We introduce the Context Substitution for Image Semantics Augmentation framework (CISA), which is focused on choosing good background images. We compare several ways to find backgrounds that match the context of the test set, including Contrastive Language–Image Pre-Training (CLIP) image retrieval and diffusion image generation. We prove that our augmentation method is effective for classification, segmentation, and object detection with different dataset complexity and different model types. The average percentage increase in accuracy across all the tasks on a fruits and vegetables recognition dataset is 4.95 % . Moreover, we show that the Fréchet Inception Distance (FID) metrics has a strong correlation with model accuracy, and it can help to choose better backgrounds without model training. The average negative correlation between model accuracy and the FID between the augmented and test datasets is 0.55 in our experiments. [ABSTRACT FROM AUTHOR]

Remote sensing is increasingly recognized as a convenient tool with a wide variety of uses in agriculture. Landsat-7 has supplied multi-spectral imagery of the Earth's surface for more than 4 years and has become an important data source for a large number of research and policy-making initiatives. Unfortunately, a scan line corrector (SLC) on Landsat-7 broke down in May 2003, which caused the loss of up to 22 percent of any given scene. We present a single-image approach based on leveraging the abilities of the deep image prior method to fill in gaps using only the corrupt image. We test the ability of deep image prior to reconstruct remote sensing scenes with different levels of corruption in them. Additionally, we compare the performance of our approach with the performance of classical single-image gap-filling methods. We demonstrate a quantitative advantage of the proposed approach compared with classical gap-filling methods. The lowest-performing restoration made by the deep image prior approach reaches 0.812 in r 2 , while the best value for the classical approaches is 0.685. We also present the robustness of deep image prior in comparing the influence of the number of corrupted pixels on the restoration results. The usage of this approach could expand the possibilities for a wide variety of agricultural studies and applications. [ABSTRACT FROM AUTHOR]

Accurate information about growing crops allows for regulating the internal stocks of agricultural products and drawing strategies for negotiating agricultural commodities on financial markets. Machine learning methods are widely implemented for crop type recognition and classification based on satellite images. However, field classification is complicated by class imbalance and aggregation of pixel-wise into field-wise forecasting. We propose here a Bayesian methodology for the aggregation of classification results. We report the comparison of class balancing techniques. We also report the comparison of classical machine learning methods and the U-Net convolutional neural network for classifying crops using a single satellite image. The best result for single-satellite-image crop classification was achieved with an overall accuracy of 77.4% and a Macro F1-score of 0.66. Bayesian aggregation for field-wise classification improved the result obtained using majority voting aggregation by 1.5%. We demonstrate here that the Bayesian aggregation approach outperforms the majority voting and averaging strategy in overall accuracy for the single-image crop classification task. [ABSTRACT FROM AUTHOR]

Estimation of terrestrial carbon balance is one of the key tasks in the understanding and prognosis of climate change impacts and the development of tools and policies according to carbon mitigation and adaptation strategies. Forest ecosystems are one of the major pools of carbon stocks affected by controversial processes influencing carbon stability. Therefore, monitoring forest ecosystems is a key to proper inventory management of resources and planning their sustainable use. In this survey, we discuss which computer vision techniques are applicable to the most important aspects of forest management actions, considering the wide availability of remote sensing (RS) data of different resolutions based both on satellite and unmanned aerial vehicle (UAV) observations. Our analysis applies to the most occurring tasks such as estimation of forest areas, tree species classification, and estimation of forest resources. Through the survey, we also provide a necessary technical background with a description of suitable data sources, algorithms' descriptions, and corresponding metrics for their evaluation. The implementation of the provided techniques into routine workflows is a significant step toward the development of systems of continuous actualization of forest data, including real-time monitoring. It is crucial for diverse purposes on both local and global scales. Among the most important are the implementation of improved forest management strategies and actions, carbon offset projects, and enhancement of the prediction accuracy of system changes under different land-use and climate scenarios. [ABSTRACT FROM AUTHOR]

Remote sensing tasks play a very important role in the domain of sensing and measuring, and can be very specific. Advances in computer vision techniques allow for the extraction of various information from remote sensing satellite imagery. This information is crucial in making quantitative and qualitative assessments for monitoring of forest clearing in protected areas for power lines, as well as for environmental analysis, in particular for making assessments of carbon footprint, which is a highly relevant task. Solving these problems requires precise segmentation of the forest mask. Although forest mask extraction from satellite data has been considered previously, no open-access applications are able to provide the high-detailed forest mask. Detailed forest masks are usually obtained using unmanned aerial vehicles (UAV) that set particular limitations such as cost and inapplicability for vast territories. In this study, we propose a novel neural network-based approach for high-detailed forest mask creation. We implement an object-based augmentation technique for a minimum amount of labeled high-detailed data. Using this augmented data we fine-tune the models, which are trained on a large forest dataset with less precise labeled masks. The provided algorithm is tested for multiple territories in Russia. The F 1 - s c o r e , for small details (such as individual trees) was improved to 0.929 compared to the baseline score of 0.856 . The developed model is available in an SAAS platform. The developed model allows a detailed and precise forest mask to be easily created, which then be used for solving various applied problems. [ABSTRACT FROM AUTHOR]

Information on forest composition, specifically tree types and their distribution, aids in timber stock calculation and can help to better understand the biodiversity in a particular region. Automatic satellite imagery analysis can significantly accelerate the process of tree type classification, which is traditionally carried out by ground-based observation. Although computer vision methods have proven their efficiency in remote sensing tasks, specific challenges arise in forestry applications. The forest inventory data often contain the tree type composition but do not describe their spatial distribution within each individual stand. Therefore, some pixels can be assigned a wrong label in the semantic segmentation task if we consider each stand to be homogeneously populated by its dominant species. Another challenge is the spatial distribution of individual stands within the study area. Classes are usually imbalanced and distributed nonuniformly that makes sampling choice more critical. This study aims to enhance tree species classification based on a neural network approach providing automatic markup adjustment and improving sampling technique. For forest species markup adjustment, we propose using a weakly supervised learning approach based on the knowledge of dominant species content within each stand. We also propose substituting the commonly used CNN sampling approach with the object-wise one to reduce the effect of the spatial distribution of forest stands. We consider four species commonly found in Russian boreal forests: birch, aspen, pine, and spruce. We use imagery from the Sentinel-2 satellite, which has multiple bands (in the visible and infrared spectra) and a spatial resolution of up to 10 meters. A data set of images for Leningrad Oblast of Russia is used to assess the methods. We demonstrate how to modify the training strategy to outperform a basic CNN approach from F1-score 0.68 to 0.76 . This approach is promising for future studies to obtain more specific information about stands composition even using incomplete data. [ABSTRACT FROM AUTHOR]

The near-infrared (NIR) spectral range (from 780 to 2500 nm) of the multispectral remote sensing imagery provides vital information for landcover classification, especially concerning vegetation assessment. Despite the usefulness of NIR, it does not always accomplish common RGB. Modern achievements in image processing via deep neural networks make it possible to generate artificial spectral information, for example, to solve the image colorization problem. In this research, we aim to investigate whether this approach can produce not only visually similar images but also an artificial spectral band that can improve the performance of computer vision algorithms for solving remote sensing tasks. We study the use of a generative adversarial network (GAN) approach in the task of the NIR band generation using only RGB channels of high-resolution satellite imagery. We evaluate the impact of a generated channel on the model performance to solve the forest segmentation task. Our results show an increase in model accuracy when using generated NIR compared to the baseline model, which uses only RGB ( 0.947 and 0.914 F1-scores, respectively). The presented study shows the advantages of generating the extra band such as the opportunity to reduce the required amount of labeled data. [ABSTRACT FROM AUTHOR]

A lot of deep learning applications are desired to be run on mobile devices. Both accuracy and inference time are meaningful for a lot of them. While the number of FLOPs is usually used as a proxy for neural network latency, it may not be the best choice. In order to obtain a better approximation of latency, the research community uses lookup tables of all possible layers for the calculation of the inference on a mobile CPU. It requires only a small number of experiments. Unfortunately, on a mobile GPU, this method is not applicable in a straightforward way and shows low precision. In this work, we consider latency approximation on a mobile GPU as a data- and hardware-specific problem. Our main goal is to construct a convenient Latency Estimation Tool for Investigation (LETI) of neural network inference and building robust and accurate latency prediction models for each specific task. To achieve this goal, we make tools that provide a convenient way to conduct massive experiments on different target devices focusing on a mobile GPU. After evaluation of the dataset, one can train the regression model on experimental data and use it for future latency prediction and analysis. We experimentally demonstrate the applicability of such an approach on a subset of the popular NAS-Benchmark 101 dataset for two different mobile GPU. [ABSTRACT FROM AUTHOR]

Usage of multispectral satellite imaging data opens vast possibilities for monitoring and quantitatively assessing properties or objects of interest on a global scale. Machine learning and computer vision (CV) approaches show themselves as promising tools for automatizing satellite image analysis. However, there are limitations in using CV for satellite data. Mainly, the crucial one is the amount of data available for model training. This paper presents a novel image augmentation approach called MixChannel that helps to address this limitation and improve the accuracy of solving segmentation and classification tasks with multispectral satellite images. The core idea is to utilize the fact that there is usually more than one image for each location in remote sensing tasks, and this extra data can be mixed to achieve the more robust performance of the trained models. The proposed approach substitutes some channels of the original training image with channels from other images of the exact location to mix auxiliary data. This augmentation technique preserves the spatial features of the original image and adds natural color variability with some probability. We also show an efficient algorithm to tune channel substitution probabilities. We report that the MixChannel image augmentation method provides a noticeable increase in performance of all the considered models in the studied forest types classification problem. [ABSTRACT FROM AUTHOR]

Currently many countries are struggling to rationalize water quality monitoring stations which is caused by economic demand. Though this process is essential indeed, the exact elements of the system to be optimized without a subsequent quality and accuracy loss still remain obscure. Therefore, accurate historical data on groundwater pollution is required to detect and monitor considerable environmental impacts. To collect such data appropriate sampling and assessment methodologies with an optimum spatial distribution augmented should be exploited. Thus, the configuration of water monitoring sampling points and the number of the points required are now considered as a fundamental optimization challenge. The paper offers and tests metaheuristic approaches for optimization of monitoring procedure and multi-factors assessment of water quality in "New Moscow" area. It is shown that the considered algorithms allow us to reduce the size of the training sample set, so that the number of points for monitoring water quality in the area can be halved. Moreover, reducing the dataset size improved the quality of prediction by 20%. The obtained results convincingly demonstrate that the proposed algorithms dramatically decrease the total cost of analysis without dampening the quality of monitoring and could be recommended for optimization purposes. [ABSTRACT FROM AUTHOR]