Search

This paper investigates data-driven solutions and parameter discovery to (2+ 1)-dimensional coupled nonlinear Schrodinger equations with variable coefficients (VC-CNLSEs), which describe transverse effects in optical fiber systems under perturbed dispersion and nonlinearity. By setting different forms of perturbation coefficients, we aim to recover the dark and anti-dark one- and two-soliton structures by employing an enhanced physics-based deep neural network algorithm, namely a physics-informed neural network (PINN). The enhanced PINN algorithm leverages the locally adaptive activation function mechanism to improve convergence speed and accuracy. In the lack of data acquisition, the PINN algorithms will enhance the capability of the neural networks by incorporating physical information into the training phase. We demonstrate that applying PINN algorithms to (2+ 1)-dimensional VC-CNLSEs requires distinct distributions of physical information. To address this, we propose a region-specific weighted loss function with the help of residual-based adaptive refinement strategy. In the meantime, we perform data-driven parameter discovery for the model equation, classified into two categories: constant coefficient discovery and variable coefficient discovery. For the former, we aim to predict the cross-phase modulation constant coefficient under varying noise intensities using enhanced PINN with a single neural network. For the latter, we employ a dual-network strategy to predict the dynamic behavior of the dispersion and non-linearity perturbation functions. Our study demonstrates that the proposed framework holds significant potential for studying high-dimensional and complex soliton dynamics in optical fiber systems., Comment: This paper involves 37 pages

Recent advancements in omnimodal learning have been achieved in understanding and generation across images, text, and speech, though mainly within proprietary models. Limited omnimodal datasets and the inherent challenges associated with real-time emotional speech generation have hindered open-source progress. To address these issues, we propose openomni, a two-stage training method combining omnimodal alignment and speech generation to develop a state-of-the-art omnimodal large language model. In the alignment phase, a pre-trained speech model is further trained on text-image tasks to generalize from vision to speech in a (near) zero-shot manner, outperforming models trained on tri-modal datasets. In the speech generation phase, a lightweight decoder facilitates real-time emotional speech through training on speech tasks and preference learning. Experiments demonstrate that openomni consistently improves across omnimodal, vision-language, and speech-language evaluations, enabling natural, emotion-rich dialogues and real-time emotional speech generation.

Adversarial Training (AT) has been widely applied to harden learning-based classifiers against adversarial evasive attacks. However, its effectiveness in identifying and strengthening vulnerable areas of the model's decision space while maintaining high performance on clean data of malware classifiers remains an under-explored area. In this context, the robustness that AT achieves has often been assessed against unrealistic or weak adversarial attacks, which negatively affect performance on clean data and are arguably no longer threats. Previous work seems to suggest robustness is a task-dependent property of AT. We instead argue it is a more complex problem that requires exploring AT and the intertwined roles played by certain factors within data, feature representations, classifiers, and robust optimization settings, as well as proper evaluation factors, such as the realism of evasion attacks, to gain a true sense of AT's effectiveness. In our paper, we address this gap by systematically exploring the role such factors have in hardening malware classifiers through AT. Contrary to recent prior work, a key observation of our research and extensive experiments confirm the hypotheses that all such factors influence the actual effectiveness of AT, as demonstrated by the varying degrees of success from our empirical analysis. We identify five evaluation pitfalls that affect state-of-the-art studies and summarize our insights in ten takeaways to draw promising research directions toward better understanding the factors' settings under which adversarial training works at best.

Extracting medication names from handwritten doctor prescriptions is challenging due to the wide variability in handwriting styles and prescription formats. This paper presents a robust method for extracting medicine names using a combination of Mask R-CNN and Transformer-based Optical Character Recognition (TrOCR) with Multi-Head Attention and Positional Embeddings. A novel dataset, featuring diverse handwritten prescriptions from various regions of Pakistan, was utilized to fine-tune the model on different handwriting styles. The Mask R-CNN model segments the prescription images to focus on the medicinal sections, while the TrOCR model, enhanced by Multi-Head Attention and Positional Embeddings, transcribes the isolated text. The transcribed text is then matched against a pre-existing database for accurate identification. The proposed approach achieved a character error rate (CER) of 1.4% on standard benchmarks, highlighting its potential as a reliable and efficient tool for automating medicine name extraction.

Deep neural network (DNN)-based joint source and channel coding is proposed for end-to-end secure image transmission against multiple eavesdroppers. Both scenarios of colluding and non-colluding eavesdroppers are considered. Instead of idealistic assumptions of perfectly known and i.i.d. source and channel distributions, the proposed scheme assumes unknown source and channel statistics. The goal is to transmit images with minimum distortion, while simultaneously preventing eavesdroppers from inferring private attributes of images. Simultaneously generalizing the ideas of privacy funnel and wiretap coding, a multi-objective optimization framework is expressed that characterizes the trade-off between image reconstruction quality and information leakage to eavesdroppers, taking into account the structural similarity index (SSIM) for improving the perceptual quality of image reconstruction. Extensive experiments over CIFAR-10 and CelebFaces Attributes (CelebA) datasets, together with ablation studies are provided to highlight the performance gain in terms of SSIM, adversarial accuracy, and cross-entropy metric compared with benchmarks. Experiments show that the proposed scheme restrains the adversarially-trained eavesdroppers from intercepting privatized data for both cases of eavesdropping a common secret, as well as the case in which eavesdroppers are interested in different secrets. Furthermore, useful insights on the privacy-utility trade-off are also provided.

The offshore wind sector relies on floating foundations for deeper waters but faces challenges from harsh conditions, nonlinear dynamics, and low-frequency resonant motions caused by second-order hydrodynamic loads. We analyze these dynamics and extract higher harmonic motions for a semisubmersible floating foundation under extreme wave conditions using experimental and numerical approaches. Two focused wave groups, with and without spreading, are considered, and experimental data is obtained from scaled physical model tests using phase-shifted input signals for harmonic decomposition of the wave responses. The responses are reproduced numerically using a novel CFD-based rigid body solver, FloatStepper, achieving good agreement. The study quantifies the effects of wave severity, spreading, and steepness on odd and even harmonics of the surge and pitch responses and mooring line tensions. A stronger sea state notably increased odd harmonics in surge and pitch. Additionally, the pitch subharmonic response, less noticeable in milder states, became apparent. Wave spreading influenced the overall response, with pronounced effects on odd and even superharmonic responses. The results reveal a front-back asymmetry in mooring line tensions, with the back lines experiencing greater tension. Increasing wavegroup amplitude caused shifts in subharmonic and superharmonic responses, transitioning from low-frequency surge-dominated behavior to coupled surge-pitch interaction. The cause of this pitch dominance is identified and discussed via CFD.

In traditional Reconfigurable Intelligent Surfaces (RIS) systems, the RIS is mounted on stationary structures like buildings, walls, or posts. They have shown promising results in enhancing the performance of wireless systems like capacity and MSE in poor channel conditions. The traditional RIS is a monolithic structure containing a large number of reflecting elements (passive or active). In this paper, we propose the idea of mounting a small number of RIS elements (usually between 2 to 4 ) on user equipment (UEs) like mobile phones, laptops, and tablets, to name a few. A joint coordinated optimization of phase shifts of all the passive RIS elements on the participating UEs is envisioned to enhance the performance of wireless communication between an intended transmitter and receiver in the MSE sense. Given that the RIS elements are mounted on the UEs, the challenging channel estimation problem with RIS is significantly simplified. For the case when there is a line-of-sight (LOS) channel and with a large number of participating RIS-mounted UEs, the LOS is converted into a multipath-rich-scattering channel even for millimeter wave and Terahertz operating ranges that enable higher spatial multiplexing gains, thereby significantly improving the MSE performance compared to traditional RIS channels. We support the above claims using simulations., Comment: Accepted for publication in 2024 Asilomar Conference on Signals, Systems, and Computers

Electric vehicles are rapidly gaining popularity as a sustainable alternative to conventional gasoline. In urban areas, chargers with different ratings can accommodate the diverse needs of electric vehicles. However, the available multiport topologies have variable switching frequencies. This paper introduces a hybrid multiport isolated DC-DC converter for urban charging stations, incorporating fast and slow charging ports with a fixed switching frequency. It provides isolation and enables soft switching on the primary side of the converter without circulating current on its secondary side. The primary side does not need feedback, which reduces complexity. The second stage generates a wide output voltage range to charge the electric vehicle battery by employing a switch. In addition, the proposed topology offers reduced component count and simple control with fixed-frequency operation. This paper provides the concept and the operation modes. Experimental results are provided to validate its features. The prototype converter achieves 96\% peak efficiency., Comment: 5 pages, 8 figures, accepted at APEC 2025 conference

One resolution of the ancient cosmic singularity, i.e., the Big Bang Singularity (BBS), is to assume an inflationary stage preceded by a long enough static state in which the universe and its physical properties would oscillate around certain equilibrium points. The early period is referred to as the Einstein Static (ES) Universe phase, which characterizes a static phase with positive spatial curvature. A stable Einstein static state can serve as a substitute for BBS, followed by an inflationary period known as the Emergent Scenario. The initial need has not been fulfilled within the context of General Relativity, prompting the investigation of modified theories of gravity. The current research aims to find such a solution within the framework of symmetric teleparallel gravity, specifically in the trendy $f(Q)$ theories. An analysis has been conducted to investigate stable solutions for both positively and negatively curved spatial FRW universes, in the presence of a perfect fluid, by utilizing various torsion-free and curvature-free affine connections. Additionally, we propose a method to facilitate an exit from a stable ES to a subsequent inflationary phase. We demonstrate that $f(Q)$ gravity theories have the ability to accurately depict the emergence of the universe., Comment: 18 pages

We investigate the quantum fate of the classical singularities that occur by gravitational collapse of a dust cloud. For this purpose, we address the quantization of a model first proposed by Georges Lemaitre in 1933. We find that the singularities can generically be avoided. This is a consequence of unitary evolution in the quantum theory, whereby the quantum dust cloud collapses, bounces at a minimal radius and re-expands., Comment: 9 pages, 3 figures, invited contribution for the conference Lemaitre 2024

This paper presents CLIP-RLDrive, a new reinforcement learning (RL)-based framework for improving the decision-making of autonomous vehicles (AVs) in complex urban driving scenarios, particularly in unsignalized intersections. To achieve this goal, the decisions for AVs are aligned with human-like preferences through Contrastive Language-Image Pretraining (CLIP)-based reward shaping. One of the primary difficulties in RL scheme is designing a suitable reward model, which can often be challenging to achieve manually due to the complexity of the interactions and the driving scenarios. To deal with this issue, this paper leverages Vision-Language Models (VLMs), particularly CLIP, to build an additional reward model based on visual and textual cues.

The quality of instruction data directly affects the performance of fine-tuned Large Language Models (LLMs). Previously, \cite{li2023one} proposed \texttt{NUGGETS}, which identifies and selects high-quality quality data from a large dataset by identifying those individual instruction examples that can significantly improve the performance of different tasks after being learnt as one-shot instances. In this work, we propose \texttt{SuperNUGGETS}, an improved variant of \texttt{NUGGETS} optimised for efficiency and performance. Our \texttt{SuperNUGGETS} uses a small language model (SLM) instead of a large language model (LLM) to filter the data for outstanding one-shot instances and refines the predefined set of tests. The experimental results show that the performance of \texttt{SuperNUGGETS} only decreases by 1-2% compared to \texttt{NUGGETS}, but the efficiency can be increased by a factor of 58. Compared to the original \texttt{NUGGETS}, our \texttt{SuperNUGGETS} has a higher utility value due to the significantly lower resource consumption.

As the capabilities of Large Language Models (LLMs) continue to advance, the field of patent processing has garnered increased attention within the natural language processing community. However, the majority of research has been concentrated on classification tasks, such as patent categorization and examination, or on short text generation tasks like patent summarization and patent quizzes. In this paper, we introduce a novel and practical task known as Draft2Patent, along with its corresponding D2P benchmark, which challenges LLMs to generate full-length patents averaging 17K tokens based on initial drafts. Patents present a significant challenge to LLMs due to their specialized nature, standardized terminology, and extensive length. We propose a multi-agent framework called AutoPatent which leverages the LLM-based planner agent, writer agents, and examiner agent with PGTree and RRAG to generate lengthy, intricate, and high-quality complete patent documents. The experimental results demonstrate that our AutoPatent framework significantly enhances the ability to generate comprehensive patents across various LLMs. Furthermore, we have discovered that patents generated solely with the AutoPatent framework based on the Qwen2.5-7B model outperform those produced by larger and more powerful LLMs, such as GPT-4o, Qwen2.5-72B, and LLAMA3.1-70B, in both objective metrics and human evaluations. We will make the data and code available upon acceptance at \url{https://github.com/QiYao-Wang/AutoPatent}., Comment: 19 pages, 7 figures

Sequential recommendation systems aim to provide personalized recommendations for users based on their interaction history. To achieve this, they often incorporate auxiliary information, such as textual descriptions of items and auxiliary tasks, like predicting user preferences and intent. Despite numerous efforts to enhance these models, they still suffer from limited personalization. To address this issue, we propose a new paradigm, which we term preference discerning. In preference dscerning, we explicitly condition a generative sequential recommendation system on user preferences within its context. To this end, we generate user preferences using Large Language Models (LLMs) based on user reviews and item-specific data. To evaluate preference discerning capabilities of sequential recommendation systems, we introduce a novel benchmark that provides a holistic evaluation across various scenarios, including preference steering and sentiment following. We assess current state-of-the-art methods using our benchmark and show that they struggle to accurately discern user preferences. Therefore, we propose a new method named Mender ($\textbf{M}$ultimodal Prefer$\textbf{en}$ce $\textbf{d}$iscern$\textbf{er}$), which improves upon existing methods and achieves state-of-the-art performance on our benchmark. Our results show that Mender can be effectively guided by human preferences even though they have not been observed during training, paving the way toward more personalized sequential recommendation systems. We will open-source the code and benchmarks upon publication., Comment: 11 pages + references and appendix

The earliest stages of star and cluster formation are hidden within dense cocoons of gas and dust, limiting their detection at optical wavelengths. With the unprecedented infrared capabilities of JWST, we can now observe dust-enshrouded star formation with $\sim$10 pc resolution out to $\sim$20 Mpc. Early findings from PHANGS-JWST suggest that 3.3 $\mu$m polycyclic aromatic hydrocarbon (PAH) emission can identify star clusters in their dust-embedded phases. Here, we extend this analysis to 19 galaxies from the PHANGS-JWST Cycle 1 Treasury Survey, providing the first characterization of compact sources exhibiting 3.3$\mu$m PAH emission across a diverse sample of nearby star-forming galaxies. We establish selection criteria, a median color threshold of F300M-F335M=0.67 at F335M=20, and identify of 1816 sources. These sources are predominantly located in dust lanes, spiral arms, rings, and galaxy centers, with $\sim$87% showing concentration indices similar to optically detected star clusters. Comparison with the PHANGS-HST catalogs suggests that PAH emission fades within $\sim$3 Myr. The H$\alpha$ equivalent width of PAH emitters is 1-2.8 times higher than that of young PHANGS-HST clusters, providing evidence that PAH emitters are on average younger. Analysis of the bright portions of luminosity functions (which should not suffer from incompleteness) shows that young dusty clusters may increase the number of optically visible $\leq$ 3 Myr-old clusters in PHANGS-HST by a factor between $\sim$1.8x-8.5x., Comment: Submitted to ApJ

Echocardiography is the most widely used imaging to monitor cardiac functions, serving as the first line in early detection of myocardial ischemia and infarction. However, echocardiography often suffers from several artifacts including sensor noise, lack of contrast, severe saturation, and missing myocardial segments which severely limit its usage in clinical diagnosis. In recent years, several machine learning methods have been proposed to improve echocardiography views. Yet, these methods usually address only a specific problem (e.g. denoising) and thus cannot provide a robust and reliable restoration in general. On the other hand, cardiac MRI provides a clean view of the heart without suffering such severe issues. However, due to its significantly higher cost, it is often only afforded by a few major hospitals, hence hindering its use and accessibility. In this pilot study, we propose a novel approach to transform echocardiography into the cardiac MRI view. For this purpose, Echo2MRI dataset, consisting of echocardiography and real cardiac MRI image pairs, is composed and will be shared publicly. A dedicated Cycle-consistent Generative Adversarial Network (Cycle-GAN) is trained to learn the transformation from echocardiography frames to cardiac MRI views. An extensive set of qualitative evaluations shows that the proposed transformer can synthesize high-quality artifact-free synthetic cardiac MRI views from a given sequence of echocardiography frames. Medical evaluations performed by a group of cardiologists further demonstrate that synthetic MRI views are indistinguishable from their original counterparts and are preferred over their initial sequence of echocardiography frames for diagnosis in 78.9% of the cases., Comment: 18 pages, 42 figures

Regarding the concepts of urban management, digital transformation, and smart cities, various issues are presented. Currently, we like to attend to location allocation problems that can be a new part of digital transformation in urban management (such as locating and placing facilities, locating and arranging centers such as aid and rescue centers, or even postal hubs, telecommunications, electronic equipment, and data centers, and routing in transportation optimization). These issues, which are seemingly simple but in practice complex, are important in urban environments, and the issue of accurate location allocation based on existing criteria directly impacts cost management, profit, efficiency, and citizen satisfaction. In recent years, researchers have used or presented various models and methods for location allocation problems, some of which will be mentioned in this article. Given the nature of these problems, which are optimization problems, this article will also examine existing research from an optimization perspective in summary. Finally, a brief conclusion will be made of the existing methods and their weaknesses, and suggestions will be made for continuing the path and improving scientific and practical research in this field., Comment: 7 pages, Keywords: Digital Transformation - Smart City - GIS - Location Allocation - Urban Management - Optimization

We present the design, modeling, and optimization of high-performance plasmonic electro-optic modulators leveraging voltage-gated carrier density in indium tin oxide (ITO) where the gated carrier density is modeled using both the Classical Drift-Diffusion (CDD) and Schr\"odinger-Poisson Coupling (SPC) methods. The latter ensures a more detailed and precise description of carrier distributions under various gate voltages which gains particular significance when applied to an epsilon-near-zero (ENZ) medium such as ITO. Combining the nanoscale confinement and field enhancement enabled by surface plasmon polaritons with the ENZ effect in ITO, modulator designs integrated with silicon waveguides and optimized for operation at {\lambda}0 = 1550 nm achieve a 3-dB bandwidth of 210 GHz, an insertion loss of 3 dB, and an extinction ratio of 5 dB for an overall length of < 4 {\mu}m as predicted by the SPC model. Our results illustrate trade-offs between high-speed modulator operation and low insertion loss, vs. extinction ratio, and the need for precise modelling of carrier distributions in ENZ materials., Comment: This manuscript has been submitted to Optics Express and is currently under review. Manuscript ID: 544216

Dependable visual drone detection is crucial for the secure integration of drones into the airspace. However, drone detection accuracy is significantly affected by domain shifts due to environmental changes, varied points of view, and background shifts. To address these challenges, we present the DrIFT dataset, specifically developed for visual drone detection under domain shifts. DrIFT includes fourteen distinct domains, each characterized by shifts in point of view, synthetic-to-real data, season, and adverse weather. DrIFT uniquely emphasizes background shift by providing background segmentation maps to enable background-wise metrics and evaluation. Our new uncertainty estimation metric, MCDO-map, features lower postprocessing complexity, surpassing traditional methods. We use the MCDO-map in our uncertainty-aware unsupervised domain adaptation method, demonstrating superior performance to SOTA unsupervised domain adaptation techniques. The dataset is available at: https://github.com/CARG-uOttawa/DrIFT.git., Comment: WACV2025

Clinical Question Answering (QA) systems enable doctors to quickly access patient information from electronic health records (EHRs). However, training these systems requires significant annotated data, which is limited due to the expertise needed and the privacy concerns associated with clinical data. This paper explores generating Clinical QA data using large language models (LLMs) in a zero-shot setting. We find that naive prompting often results in easy questions that do not reflect the complexity of clinical scenarios. To address this, we propose two prompting strategies: 1) instructing the model to generate questions that do not overlap with the input context, and 2) summarizing the input record using a predefined schema to scaffold question generation. Experiments on two Clinical QA datasets demonstrate that our method generates more challenging questions, significantly improving fine-tuning performance over baselines. We compare synthetic and gold data and find a gap between their training efficacy resulting from the quality of synthetically generated answers., Comment: Accepted to ML4H 2024 Findings

Distributed learning algorithms, such as the ones employed in Federated Learning (FL), require communication compression to reduce the cost of client uploads. The compression methods used in practice are often biased, which require error feedback to achieve convergence when the compression is aggressive. In turn, error feedback requires client-specific control variates, which directly contradicts privacy-preserving principles and requires stateful clients. In this paper, we propose Compressed Aggregate Feedback (CAFe), a novel distributed learning framework that allows highly compressible client updates by exploiting past aggregated updates, and does not require control variates. We consider Distributed Gradient Descent (DGD) as a representative algorithm and provide a theoretical proof of CAFe's superiority to Distributed Compressed Gradient Descent (DCGD) with biased compression in the non-smooth regime with bounded gradient dissimilarity. Experimental results confirm that CAFe consistently outperforms distributed learning with direct compression and highlight the compressibility of the client updates with CAFe.

We present HyperFLINT (Hypernetwork-based FLow estimation and temporal INTerpolation), a novel deep learning-based approach for estimating flow fields, temporally interpolating scalar fields, and facilitating parameter space exploration in spatio-temporal scientific ensemble data. This work addresses the critical need to explicitly incorporate ensemble parameters into the learning process, as traditional methods often neglect these, limiting their ability to adapt to diverse simulation settings and provide meaningful insights into the data dynamics. HyperFLINT introduces a hypernetwork to account for simulation parameters, enabling it to generate accurate interpolations and flow fields for each timestep by dynamically adapting to varying conditions, thereby outperforming existing parameter-agnostic approaches. The architecture features modular neural blocks with convolutional and deconvolutional layers, supported by a hypernetwork that generates weights for the main network, allowing the model to better capture intricate simulation dynamics. A series of experiments demonstrates HyperFLINT's significantly improved performance in flow field estimation and temporal interpolation, as well as its potential in enabling parameter space exploration, offering valuable insights into complex scientific ensembles.

Quantum computing offers a promising new avenue for advancing computational methods in science and engineering. In this work, we introduce the quantum asymptotic numerical method, a novel quantum nonlinear solver that combines Taylor series expansions with quantum linear solvers to efficiently address nonlinear problems. By linearizing nonlinear problems using the Taylor series, the method transforms them into sequences of linear equations solvable by quantum algorithms, thus extending the convergence region for solutions and simultaneously leveraging quantum computational advantages. Numerical tests on the quantum simulator Qiskit confirm the convergence and accuracy of the method in solving nonlinear problems. Additionally, we apply the proposed method to a beam buckling problem, demonstrating its robustness in handling strongly nonlinear problems and its potential advantages in quantum resource requirements. Furthermore, we perform experiments on a superconducting quantum processor from Quafu, successfully achieving up to 98% accuracy in the obtained nonlinear solution path. We believe this work contributes to the utility of quantum computing in scientific computing applications., Comment: 35 pages, 19 figures, 1 table, submitted to Elsevier

Resistive Random-Access Memory (ReRAM) crossbar arrays are promising candidates for in-situ matrix-vector multiplication (MVM), a frequent operation in Deep Learning algorithms. Despite their advantages, these emerging non-volatile memories are susceptible to errors due to non-idealities such as immature fabrication processes and runtime errors, which lead to accuracy degradation in Processing-in-Memory (PIM) accelerators. This paper proposes an online soft error detection and correction method in ReRAM crossbar arrays. We utilize a test input vector and Error Correcting Codes (ECCs) to detect and correct faulty columns. The proposed approach demonstrates near fault-free accuracy for Neural Networks (NNs) on MNIST and CIFAR-10 datasets, with low area overhead and power consumption compared to recent methods.

The purpose of this paper is to present a new class of operators known as polynomially hypo-EP operators, extending the notation of hypo-EP, $n$-hypo-EP, and polynomially EP. The paper explores numerous properties and characterizations of polynomially hypo-EP operators and polynomially EP operators. In addition, we characterize polynomially hypo-EP operators applying the adequate operator matrix representations. By employing these findings, we derive new characterizations of EP, hypo-EP, $n$-EP and $n$-hypo-EP.

Many challenges in science and engineering, such as drug discovery and communication network design, involve optimizing complex and expensive black-box functions across vast search spaces. Thus, it is essential to leverage existing data to avoid costly active queries of these black-box functions. To this end, while Offline Black-Box Optimization (BBO) is effective for deterministic problems, it may fall short in capturing the stochasticity of real-world scenarios. To address this, we introduce Stochastic Offline BBO (SOBBO), which tackles both black-box objectives and uncontrolled uncertainties. We propose two solutions: for large-data regimes, a differentiable surrogate allows for gradient-based optimization, while for scarce-data regimes, we directly estimate gradients under conservative field constraints, improving robustness, convergence, and data efficiency. Numerical experiments demonstrate the effectiveness of our approach on both synthetic and real-world tasks.

Reverse thinking plays a crucial role in human reasoning. Humans can reason not only from a problem to a solution but also in reverse, i.e., start from the solution and reason towards the problem. This often enhances overall reasoning performance as it enables consistency checks between their forward and backward thinking. To enable Large Language Models (LLMs) to perform reverse thinking, we introduce Reverse-Enhanced Thinking (RevThink), a framework composed of data augmentation and learning objectives. In RevThink, we augment the dataset by collecting structured forward-backward reasoning from a teacher model, consisting of: (1) the original question, (2) forward reasoning, (3) backward question, and (4) backward reasoning. We then employ three objectives to train a smaller student model in a multi-task learning fashion: (a) generate forward reasoning from a question, (b) generate a backward question from a question, and (c) generate backward reasoning from the backward question. Experiments across 12 datasets covering commonsense, math, and logical reasoning show an average 13.53% improvement over the student model's zero-shot performance and a 6.84% improvement over the strongest knowledge distillation baselines. Moreover, our method demonstrates sample efficiency -- using only 10% of the correct forward reasoning from the training data, it outperforms a standard fine-tuning method trained on 10x more forward reasoning. RevThink also exhibits strong generalization to out-of-distribution held-out datasets., Comment: 20 pages

Many natural phenomena are characterized by self-similarity, for example the symmetry of human faces, or a repetitive motif of a song. Studying of such symmetries will allow us to gain deeper insights into the underlying mechanisms of complex systems. Recognizing the importance of understanding these patterns, we propose a geometrically inspired framework to study such phenomena in artificial neural networks. To this end, we introduce \emph{CantorNet}, inspired by the triadic construction of the Cantor set, which was introduced by Georg Cantor in the $19^\text{th}$ century. In mathematics, the Cantor set is a set of points lying on a single line that is self-similar and has a counter intuitive property of being an uncountably infinite null set. Similarly, we introduce CantorNet as a sandbox for studying self-similarity by means of novel topological and geometrical complexity measures. CantorNet constitutes a family of ReLU neural networks that spans the whole spectrum of possible Kolmogorov complexities, including the two opposite descriptions (linear and exponential as measured by the description length). CantorNet's decision boundaries can be arbitrarily ragged, yet are analytically known. Besides serving as a testing ground for complexity measures, our work may serve to illustrate potential pitfalls in geometry-ignorant data augmentation techniques and adversarial attacks., Comment: Accepted at the NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2024

In-memory computing (IMC) offloads parts of the computations to memory to fulfill the performance and energy demands of applications such as neuromorphic computing, machine learning, and image processing. Fortunately, the main features that stochastic computing (SC) and IMC share, which are low computation complexity and high bit-parallel computation capability, promise great potential for integrating SC and IMC. In this paper, we exploit this potential by using stochastic computation as an approximation method to present effective in-memory computations with a good trade-off among design parameters. To this end, first, commonly used stochastic arithmetic operations of applications are effectively implemented using the primitive logic gates of the IMC method. Next, the in-memory scheduling and mapping of applications are obtained efficiently by a proposed algorithm. This algorithm reduces the computation latency by enabling intra-subarray parallelism while considering the IMC method constraints. Subsequently, a bit-parallel stochastic IMC architecture, Stoch-IMC, is presented that enables bit parallelization of stochastic computations over memory subarrays/banks. To evaluate Stoch-IMC's effectiveness, various analyses were conducted. Results show average performance improvements of 135.7X and 124.2X across applications compared to binary IMC and related in-memory SC methods, respectively. The results also demonstrate an average energy reduction of 1.5X compared to binary IMC, with limited energy overhead relative to the in-memory SC method. Furthermore, the results reveal average lifetime improvements of 4.9X and 216.3X over binary IMC and in-memory SC methods, respectively, along with high bitflip tolerance.