Search

Second-order methods are widely adopted to improve the convergence rate of learning algorithms. In federated learning (FL), these methods require the clients to share their local Hessian matrices with the parameter server (PS), which comes at a prohibitive communication cost. A classical solution to this issue is to approximate the global Hessian matrix from the first-order information. Unlike in idealized networks, this solution does not perform effectively in over-the-air FL settings, where the PS receives noisy versions of the local gradients. This paper introduces a novel second-order FL framework tailored for wireless channels. The pivotal innovation lies in the PS's capability to directly estimate the global Hessian matrix from the received noisy local gradients via a non-parametric method: the PS models the unknown Hessian matrix as a Gaussian process, and then uses the temporal relation between the gradients and Hessian along with the channel model to find a stochastic estimator for the global Hessian matrix. We refer to this method as Gaussian process-based Hessian modeling for wireless FL (GP-FL) and show that it exhibits a linear-quadratic convergence rate. Numerical experiments on various datasets demonstrate that GP-FL outperforms all classical baseline first and second order FL approaches., Comment: The paper is submitted to IEEE Transactions on Signal Processing

We present some regularity results on the gradient of the weak or entropic-renormalized solution $u$ to the homogeneous Dirichlet problem for the quasilinear equations of the form \begin{equation*}\label{p-laplacian_eq} -{\rm div~}(|\nabla u|^{p-2}\nabla u)+V(x;u)=f, \end{equation*} where $\Omega$ is a bounded smooth domain of $\mathbb R^n$, $V$ is a nonlinear potential and $f$ belongs to non-standard spaces like Lorentz-Zygmund spaces. Moreover, we collect some well-known and new results for identifying some interpolation spaces and enrich some contents with details.

The statistical analysis of land clutter for Synthetic Aperture Radar (SAR) imaging has become an increasingly important subject for research and investigation. It is also absolutely necessary for designing robust algorithms capable of performing the task of target detection in the background clutter. Any attempt to extract the energy of the desired targets from the land clutter requires complete knowledge of the statistical properties of the background clutter. In this paper, the spatial as well as the temporal characteristics of the land clutter are studied. Since the data for each image has been collected based on a different aspect angle; therefore, the temporal analysis contains variation in the aspect angle. Consequently, the temporal analysis includes the characteristics of the radar cross section with respect to the aspect angle based on which the data has been collected. In order to perform the statistical analysis, several well-known and relevant distributions, namely, Weibull, Log-normal, Gamma, and Rayleigh are considered as prime candidates to model the land clutter. The goodness-of-fit test is based on the Kullback-Leibler (KL) Divergence metric. The detailed analysis presented in this paper demonstrates that the Weibull distribution is a more accurate fit for the temporal-aspect-angle statistical analysis while the Rayleigh distribution models the spatial characteristics of the background clutter with higher accuracy. Finally, based on the aforementioned statistical analyses and by utilizing the Constant False Alarm Rate (CFAR) algorithm, we perform target detection in land clutter. The overall verification of the analysis is performed by exploiting the Moving and Stationary Target Acquisition and Recognition (MSTAR) data-set, which has been collected in spotlight mode at X-band, and the results are presented., Comment: arXiv admin note: substantial text overlap with arXiv:2409.02155

This article proposes a source-independent method for detecting faults along Transmission Lines (TL) to reduce the protection issues arising from Inverter-Based Resources (IBRs). In the proposed method, high-frequency waves are sent from either end of a TL, and the amplitudes of the receiving waves at the other end are measured. Faults change the characteristics of TLs. Therefore, the amplitudes of the receiving waves differ when a fault occurs. Closed-form formulations are developed for describing the receiving waves before and during the faults. These formulations indicate that at least one of the receiving waves is reduced after fault inception. Therefore, faults can be detected by identifying a decrease in one of the receiving waves. To evaluate the performance of the proposed method, EMTP-RV is utilized for performing simulations. Additionally, laboratory experiments are conducted for further evaluation of the proposed method. The simulation and experimental results demonstrate that the proposed method is able to detect faults along TLs regardless of the sources supplying the grid., Comment: Pre-print of a manuscript submitted to the IEEE Trans. on Power Deliver. Based on IEEE rules, publishing pre-prints without final pagination and edits is allowed

Quantization is a common approach to mitigate the communication cost of federated learning (FL). In practice, the quantized local parameters are further encoded via an entropy coding technique, such as Huffman coding, for efficient data compression. In this case, the exact communication overhead is determined by the bit rate of the encoded gradients. Recognizing this fact, this work deviates from the existing approaches in the literature and develops a novel quantized FL framework, called \textbf{r}ate-\textbf{c}onstrained \textbf{fed}erated learning (RC-FED), in which the gradients are quantized subject to both fidelity and data rate constraints. We formulate this scheme, as a joint optimization in which the quantization distortion is minimized while the rate of encoded gradients is kept below a target threshold. This enables for a tunable trade-off between quantization distortion and communication cost. We analyze the convergence behavior of RC-FED, and show its superior performance against baseline quantized FL schemes on several datasets.

In this paper, the challenging task of target detection in sea clutter is addressed. We analyze the statistical properties of the signals which have been received from the scene and based on that, we model the amplitude of the signals that have been reflected from the sea clutter according to several well-known probability distributions. Next, by exploiting the Kullback-Leibler (KL) divergence metric as a goodness-of-fit test, we will demonstrate that among the proposed probability distributions, the Weibull distribution can model the statistical properties of the sea clutter with higher accuracy. Subsequently, we utilize the aforementioned information to design an adaptive threshold based on the Constant False Alarm Rate (CFAR) algorithm to detect the energy of the targets which have been buried in the sea clutter. Thorough analysis of the experimental data gathered from the Canadian RADARSAT-1 satellite demonstrates the overall effectiveness of the proposed method., Comment: arXiv admin note: text overlap with arXiv:2008.07457

Magnetooptical Kerr effect in meta surface based structure

The reconstructed images from the Synthetic Aperture Radar (SAR) data suffer from multiplicative noise as well as low contrast level. These two factors impact the quality of the SAR images significantly and prevent any attempt to extract valuable information from the processed data. The necessity for mitigating these effects in the field of SAR imaging is of high importance. Therefore, in this paper, we address the aforementioned issues and propose a technique to handle these shortcomings simultaneously. In fact, we combine the de-noising and contrast enhancement processes into a unified algorithm. The image enhancement is performed based on the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique. The verification of the proposed algorithm is performed by experimental results based on the data that has been collected from the European Space Agency's ERS-2 satellite which operates in strip-map mode.

Recently, it was shown that the role of the teacher in knowledge distillation (KD) is to provide the student with an estimate of the true Bayes conditional probability density (BCPD). Notably, the new findings propose that the student's error rate can be upper-bounded by the mean squared error (MSE) between the teacher's output and BCPD. Consequently, to enhance KD efficacy, the teacher should be trained such that its output is close to BCPD in MSE sense. This paper elucidates that training the teacher model with MSE loss equates to minimizing the MSE between its output and BCPD, aligning with its core responsibility of providing the student with a BCPD estimate closely resembling it in MSE terms. In this respect, through a comprehensive set of experiments, we demonstrate that substituting the conventional teacher trained with cross-entropy loss with one trained using MSE loss in state-of-the-art KD methods consistently boosts the student's accuracy, resulting in improvements of up to 2.6\%., Comment: The paper was accepted at ECCV2024

Understanding the mechanisms behind decisions taken by large foundation models in sequential decision making tasks is critical to ensuring that such systems operate transparently and safely. In this work, we perform exploratory analysis on the Video PreTraining (VPT) Minecraft playing agent, one of the largest open-source vision-based agents. We aim to illuminate its reasoning mechanisms by applying various interpretability techniques. First, we analyze the attention mechanism while the agent solves its training task - crafting a diamond pickaxe. The agent pays attention to the last four frames and several key-frames further back in its six-second memory. This is a possible mechanism for maintaining coherence in a task that takes 3-10 minutes, despite the short memory span. Secondly, we perform various interventions, which help us uncover a worrying case of goal misgeneralization: VPT mistakenly identifies a villager wearing brown clothes as a tree trunk when the villager is positioned stationary under green tree leaves, and punches it to death., Comment: Mechanistic Interpretability Workshop at ICML 2024

This paper proposes a source-independent method for the detection and classification of faults along Transmission Lines (TLs). It aims to reduce the protection issues arising from Inverter-Based Resources (IBRs). Inspired by Power Line Communication (PLC), the proposed method utilizes high-frequency carrier waves which are sent from either side of a TL over each phase. As faults disrupt the propagation of carriers, the receiving carrier waves before and during faults exhibit differences. Based on this principle, the proposed method continuously compares the receiving carrier waves with a short history of them to detect and classify faults. The performance of the proposed method was evaluated using EMTP-RV and MATLAB, and compared to traditional phasor-based distance relays. The simulation results confirm the capability of the proposed method in detection and classification of different faults regardless of power sources types., Comment: 5 pages. This is the preprint of a conference paper which is accepted for presentation at IEEE PES-GM2024. Based on the IEEE regulations, publishing preprints without the full format of IEEE (index number, DOI, etc) is allowed

This paper proposes a recursive method for integrity verification of measured transformer currents, which is suitable for the modular development of Hardware-In-the-Loop Digital Twins (HIL DTs). The Differential Equations (DEs) describing transformer transients are relatively complex, requiring the use of numerical DE solvers with small time steps. This implies that replicating transformers with HIL DTs requires a continuous flow of accurate measurement samples with a high sampling rate. The proposed method utilizes the Adaptive Extended Kalman Filter to estimate the parameters of transformer currents which can be non-sinusoidal during transients such as transformer energization. Then, after evaluating the validity of estimations, the proposed method utilizes the estimated parameters to reconstruct noiseless measurements with desirable sampling rates. The performance of the proposed method is evaluated using EMTP-RV for simulating transformers and Matlab for executing the proposed method. The simulation results demonstrate that the method is able to accurately estimate and closely track the current measurements., Comment: This is a 10-page preprint of my work

In this paper, we present a unique $\text {2D}$ high resolution, compact, low-cost, low-weight, and highly accurate millimeter wave imagery system capable of operating in all weather conditions. We describe millimeter wave imaging process in detail and present several novel signal processing methods with their applications. To create the array, we utilize the Synthetic Aperture Radar (SAR) concept. The imagery system presented in this paper, can strongly compete with Lidar systems as the resolution limit is at the same level. Furthermore, in contrast to the Lidar systems, our imagery system can operate in heavy rain and dense fog and produce high quality images. Finally, we utilize our wide-band custom-made Frequency Modulated Continuous Wave (FMCW) radar, which operates at W-band with $\text {33 GHz}$ bandwidth, for data collection and present the results.

Structural modularity is a pervasive feature of biological neural networks, which have been linked to several functional and computational advantages. Yet, the use of modular architectures in artificial neural networks has been relatively limited despite early successes. Here, we explore the performance and functional dynamics of a modular network trained on a memory task via an iterative growth curriculum. We find that for a given classical, non-modular recurrent neural network (RNN), an equivalent modular network will perform better across multiple metrics, including training time, generalizability, and robustness to some perturbations. We further examine how different aspects of a modular network's connectivity contribute to its computational capability. We then demonstrate that the inductive bias introduced by the modular topology is strong enough for the network to perform well even when the connectivity within modules is fixed and only the connections between modules are trained. Our findings suggest that gradual modular growth of RNNs could provide advantages for learning increasingly complex tasks on evolutionary timescales, and help build more scalable and compressible artificial networks.

Adversarial training (AT) is a popular method for training robust deep neural networks (DNNs) against adversarial attacks. Yet, AT suffers from two shortcomings: (i) the robustness of DNNs trained by AT is highly intertwined with the size of the DNNs, posing challenges in achieving robustness in smaller models; and (ii) the adversarial samples employed during the AT process exhibit poor generalization, leaving DNNs vulnerable to unforeseen attack types. To address these dual challenges, this paper introduces adversarial training via adaptive knowledge amalgamation of an ensemble of teachers (AT-AKA). In particular, we generate a diverse set of adversarial samples as the inputs to an ensemble of teachers; and then, we adaptively amalgamate the logtis of these teachers to train a generalized-robust student. Through comprehensive experiments, we illustrate the superior efficacy of AT-AKA over existing AT methods and adversarial robustness distillation techniques against cutting-edge attacks, including AutoAttack.