Search

Your search keyword '"Hanin, A."' showing total 7,862 results
Start Over Author "Hanin, A."
7,862 results on '"Hanin, A."'

1. What Do Speech Foundation Models Not Learn About Speech?

Author

Waheed, Abdul, Atwany, Hanin, Raj, Bhiksha, and Singh, Rita

Subjects
Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing
Abstract

Understanding how speech foundation models capture non-verbal cues is crucial for improving their interpretability and adaptability across diverse tasks. In our work, we analyze several prominent models such as Whisper, Seamless, Wav2Vec, HuBERT, and Qwen2-Audio focusing on their learned representations in both paralinguistic and non-paralinguistic tasks from the Dynamic-SUPERB benchmark. Our study addresses three key questions: (1) What non-verbal cues (e.g., speaker intent, emotion, environmental context) are captured? (2) How are these cues represented across different layers of the models? and (3) To what extent can these representations be effectively adapted to downstream tasks? To answer these questions, we first evaluate the models in a zero-shot setting, followed by fine-tuning on layer-wise features extracted from these models. Our results provide insights into the models' capacity for generalization, the characteristics of their layer-wise representations, and the degree of transformation required for downstream task adaptation. Our findings suggest that some of these models perform well on various tasks in zero-shot settings, despite not being explicitly trained for those tasks. We also observe that zero-shot performance correlates with better-learned representations. The analysis of layer-wise features demonstrates that some models exhibit a convex relationship between the separability of the learned representations and model depth, with different layers capturing task-specific features., Comment: 20 Pages

Published
2024

2. Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Author

Razin, Noam, Malladi, Sadhika, Bhaskar, Adithya, Chen, Danqi, Arora, Sanjeev, and Hanin, Boris

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Statistics - Machine Learning
Abstract

Direct Preference Optimization (DPO) and its variants are increasingly used for aligning language models with human preferences. Although these methods are designed to teach a model to generate preferred responses more frequently relative to dispreferred responses, prior work has observed that the likelihood of preferred responses often decreases during training. The current work sheds light on the causes and implications of this counter-intuitive phenomenon, which we term likelihood displacement. We demonstrate that likelihood displacement can be catastrophic, shifting probability mass from preferred responses to responses with an opposite meaning. As a simple example, training a model to prefer $\texttt{No}$ over $\texttt{Never}$ can sharply increase the probability of $\texttt{Yes}$. Moreover, when aligning the model to refuse unsafe prompts, we show that such displacement can unintentionally lead to unalignment, by shifting probability mass from preferred refusal responses to harmful responses (e.g., reducing the refusal rate of Llama-3-8B-Instruct from 74.4% to 33.4%). We theoretically characterize that likelihood displacement is driven by preferences that induce similar embeddings, as measured by a centered hidden embedding similarity (CHES) score. Empirically, the CHES score enables identifying which training samples contribute most to likelihood displacement in a given dataset. Filtering out these samples effectively mitigated unintentional unalignment in our experiments. More broadly, our results highlight the importance of curating data with sufficiently distinct preferences, for which we believe the CHES score may prove valuable., Comment: Code available at https://github.com/princeton-nlp/unintentional-unalignment

Published
2024

3. Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design

Author

Davis, Jared Quincy, Hanin, Boris, Chen, Lingjiao, Bailis, Peter, Stoica, Ion, and Zaharia, Matei

Subjects
Computer Science - Artificial Intelligence
Abstract

As practitioners seek to surpass the current reliability and quality frontier of monolithic models, Compound AI Systems consisting of many language model inference calls are increasingly employed. In this work, we construct systems, which we call Networks of Networks (NoNs) organized around the distinction between generating a proposed answer and verifying its correctness, a fundamental concept in complexity theory that we show empirically extends to Language Models (LMs). We introduce a verifier-based judge NoN with K generators, an instantiation of "best-of-K" or "judge-based" compound AI systems. Through experiments on synthetic tasks such as prime factorization, and core benchmarks such as the MMLU, we demonstrate notable performance gains. For instance, in factoring products of two 3-digit primes, a simple NoN improves accuracy from 3.7\% to 36.6\%. On MMLU, a verifier-based judge construction with only 3 generators boosts accuracy over individual GPT-4-Turbo calls by 2.8\%. Our analysis reveals that these gains are most pronounced in domains where verification is notably easier than generation--a characterization which we believe subsumes many reasoning and procedural knowledge tasks, but doesn't often hold for factual and declarative knowledge-based settings. For mathematical and formal logic reasoning-based subjects of MMLU, we observe a 5-8\% or higher gain, whilst no gain on others such as geography and religion. We provide key takeaways for ML practitioners, including the importance of considering verification complexity, the impact of witness format on verifiability, and a simple test to determine the potential benefit of this NoN approach for a given problem distribution. This work aims to inform future research and practice in the design of compound AI systems.

Published
2024

4. Bayesian Inference with Deep Weakly Nonlinear Networks

Author

Hanin, Boris and Zlokapa, Alexander

Subjects
Statistics - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Mathematics - Probability, Physics - Data Analysis, Statistics and Probability
Abstract

We show at a physics level of rigor that Bayesian inference with a fully connected neural network and a shaped nonlinearity of the form $\phi(t) = t + \psi t^3/L$ is (perturbatively) solvable in the regime where the number of training datapoints $P$ , the input dimension $N_0$, the network layer widths $N$, and the network depth $L$ are simultaneously large. Our results hold with weak assumptions on the data; the main constraint is that $P < N_0$. We provide techniques to compute the model evidence and posterior to arbitrary order in $1/N$ and at arbitrary temperature. We report the following results from the first-order computation: 1. When the width $N$ is much larger than the depth $L$ and training set size $P$, neural network Bayesian inference coincides with Bayesian inference using a kernel. The value of $\psi$ determines the curvature of a sphere, hyperbola, or plane into which the training data is implicitly embedded under the feature map. 2. When $LP/N$ is a small constant, neural network Bayesian inference departs from the kernel regime. At zero temperature, neural network Bayesian inference is equivalent to Bayesian inference using a data-dependent kernel, and $LP/N$ serves as an effective depth that controls the extent of feature learning. 3. In the restricted case of deep linear networks ($\psi=0$) and noisy data, we show a simple data model for which evidence and generalization error are optimal at zero temperature. As $LP/N$ increases, both evidence and generalization further improve, demonstrating the benefit of depth in benign overfitting.

Published
2024

5. Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems

Author

Chen, Lingjiao, Davis, Jared Quincy, Hanin, Boris, Bailis, Peter, Stoica, Ion, Zaharia, Matei, and Zou, James

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Electrical Engineering and Systems Science - Systems and Control
Abstract

Many recent state-of-the-art results in language tasks were achieved using compound systems that perform multiple Language Model (LM) calls and aggregate their responses. However, there is little understanding of how the number of LM calls - e.g., when asking the LM to answer each question multiple times and taking a majority vote - affects such a compound system's performance. In this paper, we initiate the study of scaling properties of compound inference systems. We analyze, theoretically and empirically, how the number of LM calls affects the performance of Vote and Filter-Vote, two of the simplest compound system designs, which aggregate LM responses via majority voting, optionally applying LM filters. We find, surprisingly, that across multiple language tasks, the performance of both Vote and Filter-Vote can first increase but then decrease as a function of the number of LM calls. Our theoretical results suggest that this non-monotonicity is due to the diversity of query difficulties within a task: more LM calls lead to higher performance on "easy" queries, but lower performance on "hard" queries, and non-monotone behavior can emerge when a task contains both types of queries. This insight then allows us to compute, from a small number of samples, the number of LM calls that maximizes system performance, and define an analytical scaling model for both systems. Experiments show that our scaling model can accurately predict the performance of Vote and Filter-Vote systems and thus find the optimal number of LM calls to make.

Published
2024

6. Principled Architecture-aware Scaling of Hyperparameters

Author

Chen, Wuyang, Wu, Junru, Wang, Zhangyang, and Hanin, Boris

Subjects
Computer Science - Machine Learning
Abstract

Training a high-quality deep neural network requires choosing suitable hyperparameters, which is a non-trivial and expensive process. Current works try to automatically optimize or design principles of hyperparameters, such that they can generalize to diverse unseen scenarios. However, most designs or optimization methods are agnostic to the choice of network structures, and thus largely ignore the impact of neural architectures on hyperparameters. In this work, we precisely characterize the dependence of initializations and maximal learning rates on the network architecture, which includes the network depth, width, convolutional kernel size, and connectivity patterns. By pursuing every parameter to be maximally updated with the same mean squared change in pre-activations, we can generalize our initialization and learning rates across MLPs (multi-layer perception) and CNNs (convolutional neural network) with sophisticated graph topologies. We verify our principles with comprehensive experiments. More importantly, our strategy further sheds light on advancing current benchmarks for architecture design. A fair comparison of AutoML algorithms requires accurate network rankings. However, we demonstrate that network rankings can be easily changed by better training networks in benchmarks with our architecture-aware learning rates and initialization.

Published
2024

10. Origin and evolution of the bread wheat D genome

Author

Cavalet-Giorsa, Emile, González-Muñoz, Andrea, Athiyannan, Naveenkumar, Holden, Samuel, Salhi, Adil, Gardener, Catherine, Quiroz-Chávez, Jesús, Rustamova, Samira M., Elkot, Ahmed Fawzy, Patpour, Mehran, Rasheed, Awais, Mao, Long, Lagudah, Evans S., Periyannan, Sambasivam K., Sharon, Amir, Himmelbach, Axel, Reif, Jochen C., Knauft, Manuela, Mascher, Martin, Stein, Nils, Chayut, Noam, Ghosh, Sreya, Perovic, Dragan, Putra, Alexander, Perera, Ana B., Hu, Chia-Yi, Yu, Guotai, Ahmed, Hanin Ibrahim, Laquai, Konstanze D., Rivera, Luis F., Chen, Renjie, Wang, Yajun, Gao, Xin, Liu, Sanzhen, Raupp, W. John, Olson, Eric L., Lee, Jong-Yeol, Chhuneja, Parveen, Kaur, Satinder, Zhang, Peng, Park, Robert F., Ding, Yi, Liu, Deng-Cai, Li, Wanlong, Nasyrova, Firuza Y., Dvorak, Jan, Abbasi, Mehrdad, Li, Meng, Kumar, Naveen, Meyer, Wilku B., Boshoff, Willem H. P., Steffenson, Brian J., Matny, Oadi, Sharma, Parva K., Tiwari, Vijay K., Grewal, Surbhi, Pozniak, Curtis J., Chawla, Harmeet Singh, Ens, Jennifer, Dunning, Luke T., Kolmer, James A., Lazo, Gerard R., Xu, Steven S., Gu, Yong Q., Xu, Xianyang, Uauy, Cristobal, Abrouk, Michael, Bougouffa, Salim, Brar, Gurcharn S., Wulff, Brande B. H., and Krattinger, Simon G.

Published
2024
Full Text
View/download PDF

13. Black Seed Oil-Based Curcumin Nanoformulations Ameliorated Cuprizone-Induced Demyelination in the Mouse Hippocampus

Author

Alam, Mohammad Zubair, Bagabir, Hala Abubaker, Zaher, Mohammad Alameen Faisal, Alqurashi, Thamer M. A., Alghamdi, Badrah S., Kazi, Mohsin, Ashraf, Ghulam Md, Alshahrany, Gadah Ali, Alzahrani, Noor Ahmed, Bakhalgi, Rafal Mohammed, Juweiriya, Al-Thepyani, Mona, AboTaleb, Hanin Abdulbaset, Aldhahri, Rahaf Saeed, El-Aziz, Gamal Said Abd, Al-Abbasi, Fahad A., Eibani, Loay Khaled, Alzahrani, Faisal Jaman, and Khan, Mohd Sajjad Ahmad

Published
2024
Full Text
View/download PDF

15. Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit

Author

Bordelon, Blake, Noci, Lorenzo, Li, Mufan Bill, Hanin, Boris, and Pehlevan, Cengiz

Subjects
Statistics - Machine Learning, Condensed Matter - Disordered Systems and Neural Networks, Computer Science - Artificial Intelligence, Computer Science - Machine Learning
Abstract

The cost of hyperparameter tuning in deep learning has been rising with model sizes, prompting practitioners to find new tuning methods using a proxy of smaller networks. One such proposal uses $\mu$P parameterized networks, where the optimal hyperparameters for small width networks transfer to networks with arbitrarily large width. However, in this scheme, hyperparameters do not transfer across depths. As a remedy, we study residual networks with a residual branch scale of $1/\sqrt{\text{depth}}$ in combination with the $\mu$P parameterization. We provide experiments demonstrating that residual architectures including convolutional ResNets and Vision Transformers trained with this parameterization exhibit transfer of optimal hyperparameters across width and depth on CIFAR-10 and ImageNet. Furthermore, our empirical findings are supported and motivated by theory. Using recent developments in the dynamical mean field theory (DMFT) description of neural network learning dynamics, we show that this parameterization of ResNets admits a well-defined feature learning joint infinite-width and infinite-depth limit and show convergence of finite-size network dynamics towards this limit.

Published
2023

16. Les Houches Lectures on Deep Learning at Large & Infinite Width

Author

Bahri, Yasaman, Hanin, Boris, Brossollet, Antonin, Erba, Vittorio, Keup, Christian, Pacelli, Rosalba, and Simon, James B.

Subjects
Statistics - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Machine Learning, High Energy Physics - Theory, Mathematics - Probability
Abstract

These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinite-width limit; and perturbative and non-perturbative treatments of large but finite-width networks, at initialization and after training., Comment: These are notes from lectures delivered by Yasaman Bahri and Boris Hanin at the 2022 Les Houches Summer School on Statistics Physics and Machine Learning and a first version of them were transcribed by Antonin Brossollet, Vittorio Erba, Christian Keup, Rosalba Pacelli, James B. Simon

Published
2023

17. Exploring seed characteristics and performance through advanced physico-chemical techniques

Author

Dhanalakshmi Vadivel, Rania Djemal, Jessica García, Andrea Pagano, Rahma Trabelsi, Maroua Gdoura-Ben Amor, Safa Charfeddine, Siwar Ghanmi, Ibtisem Khalifa, Mariem Rekik, Fatma Amor, Chantal Ebel, Radhouane Gdoura, Amine Elleuch, Alma Balestrazzi, Anca Macovei, Moez Hanin, and Daniele Dondi

Subjects
Seed, Physico-chemical profile, Thermogravimetric analysis, Electron paramagnetic resonance, High-performance liquid chromatography, Principal component analysis, Medicine, Science
Abstract

Abstract Simple physico-chemical techniques can be used to evaluate the composition, structure, and characteristics of plant seeds to determine their viability, quality, and possible uses in agriculture. Advanced analytical techniques, including thermo gravimetric analysis (TGA), electron paramagnetic resonance (EPR), and high-pressure liquid chromatography (HPLC), provide completely new insights and more precise information. They can be integrated to build up seed quality profiles, with great advantage to assess water content, organic compounds, and inorganic metals without the need to carry out many extraction procedures, as requested by more conventional methods. In this study, seed lots from three different plant species such as Triticum turgidum L. subsp. durum (wheat), Trigonella foenum graecum L. (trigonella or fenugreek), and Atriplex halimus L. (saltbush or sea orach) have been used to test the potential of TGA, EPR, and HPLC to discriminate between seed-specific features. A key finding of this study is that HPLC is essential in Principal Component Analysis (PCA) because various seeds (from the same species or other species) may contain compounds with varying polarity groups. The reported data confirm the efficacy of this approach. These data, fully available for other users, are coherently constructed and provide a proof of concept for future seed quality control studies.

Published
2024
Full Text
View/download PDF

18. What methods are currently available for incorporating implementation considerations within the economic evaluation of health technologies? A scoping review

Author

Robert Heggie, Kathleen Boyd, Hanin Kamaruzaman, and Olivia Wu

Subjects
Implementation, Economic evaluation, Health technology assessment, Public aspects of medicine, RA1-1270
Abstract

Abstract Background When clinically effective, cost-effective health interventions are not fully implemented in clinical practice, population health suffers. Economic factors are among the most commonly cited reasons for suboptimal implementation. Despite this, implementation and economic evaluation are not routinely performed in conjunction with one another. This review sought to identify and describe what methods are available for researchers to incorporate implementation within economic evaluation, how these methods differ, when they should be used, and where gaps remain. Methods We conducted a scoping review using systematic methods. A pearl-growing approach was used to identify studies. References and citations were identified using Web of Science and Scopus. We included for review any study that contained terms relating to economic evaluation and a series of implementation-related terms in the title or abstract. The search was conducted and validated using two independent researchers. Results Our review identified 42 unique studies that included a methodology for combining implementation and economic evaluation. The methods identified could be categorized into four broad themes: (i) policy cost–effectiveness approach (11 studies), (ii) value of information and value of implementation approach (16 studies), (iii) mixed methods approach (6 studies), and (iv) costing approach (9 studies). We identified a trend over time from methods that adopted the policy cost–effectiveness approach to methods that considered the trade-off between the value of information and value of implementation. More recently, mixed methods approaches to incorporate economic evaluation and implementation have been developed, alongside methods to define, measure and cost individual components of the implementation process for use in economic evaluation. Conclusion Our review identified a range of methods currently available for researchers considering implementation alongside economic evaluation. There is no single method or tool that can incorporate all the relevant issues to fully incorporate implementation within an economic evaluation. Instead, there are a suite of tools available, each of which can be used to answer a specific question relating to implementation. Researchers, reimbursement agencies and national and local decision-makers need to consider how best to utilize these tools to improve implementation.

Published
2024
Full Text
View/download PDF

19. Association between $$PM_{10}$$ P M 10 and respiratory diseases admission in peninsula Malaysia during haze

Author

Siti Wafiah Hanin Mohd Zulkifli, Humaida Banu Samsudin, and Noriza Majid

Subjects
Particulate matter ( $$PM_{10}$$ P M 10 ), Hospital admissions, Respiratory diseases, Relative risk, Medicine, Science
Abstract

Abstract Numerous studies have been conducted in other countries on the health effects of exposure to particulate matter with a diameter of 10 microns or less $$PM_{10}$$ P M 10 , but little research has been conducted in Malaysia, particularly during the haze season. This study intends to investigate how exposure of $$PM_{10}$$ P M 10 influenced hospital admissions for respiratory diseases during the haze period in peninsula Malaysia and it was further stratified by age group, gender and respiratory diseases categories. The study includes data from all patients with respiratory diseases in 92 government hospitals, as well as $$PM_{10}$$ P M 10 concentration and meteorological data from 92 monitoring stations in Peninsula Malaysia starting from 1st January 2000 to 31st December 2019. A quasi-poison time series regression with distributed lag nonlinear model (DLNM) was employed in this study to examine the relationship between exposure of $$PM_{10}$$ P M 10 and hospital admissions for respiratory diseases during the haze period. Haze period for this study has been defined from June to September each year. According to the findings of this study, $$PM_{10}$$ P M 10 was positively associated with hospitalisation of respiratory disease within 30 lag days under various lag patterns, with lag 25 showing the strongest association (RR = 1.001742, CI 1.001029,1.002456). Using median as a reference, it was discovered that females were more likely than males to be hospitalized for $$PM_{10}$$ P M 10 exposure. Working age group will be the most affected by the increase in $$PM_{10}$$ P M 10 exposure with a significant cumulative RR from lag 010 to lag 030. The study found that $$PM_{10}$$ P M 10 had a significant influence on respiratory hospitalisation in peninsula Malaysia, particularly for lung diseases caused by external agents(CD5). Therefore, it is important to implement effective intervention measures to control $$PM_{10}$$ P M 10 and reduce the burden of respiratory disease admissions.

Published
2024
Full Text
View/download PDF

20. Digital Vocal Biomarker of Smoking Status Using Ecological Audio Recordings: Results from the Colive Voice Study

Author

Hanin Ayadi, Abir Elbéji, Vladimir Despotovic, and Guy Fagherazzi

Subjects
public health, tobacco, smoking, vocal biomarkers, machine learning, Biology (General), QH301-705.5
Abstract

Introduction: The complex health, social, and economic consequences of tobacco smoking underscore the importance of incorporating reliable and scalable data collection on smoking status and habits into research across various disciplines. Given that smoking impacts voice production, we aimed to develop a gender and language-specific vocal biomarker of smoking status. Methods: Leveraging data from the Colive Voice study, we used statistical analysis methods to quantify the effects of smoking on voice characteristics. Various voice feature extraction methods combined with machine learning algorithms were then used to produce a gender and language-specific (English and French) digital vocal biomarker to differentiate smokers from never-smokers. Results: A total of 1,332‬ participants were included after propensity score matching (mean age = 43.6 [13.65], 64.41% are female, 56.68% are English speakers, 50% are smokers and 50% are never-smokers). We observed differences in voice features distribution: for women, the fundamental frequency F0, the formants F1, F2, and F3 frequencies and the harmonics-to-noise ratio were lower in smokers compared to never-smokers (p < 0.05) while for men no significant disparities were noted between the two groups. The accuracy and AUC of smoking status prediction reached 0.71 and 0.76, respectively, for the female participants, and 0.65 and 0.68, respectively, for the male participants. Conclusion: We have shown that voice features are impacted by smoking. We have developed a novel digital vocal biomarker that can be used in clinical and epidemiological research to assess smoking status in a rapid, scalable, and accurate manner using ecological audio recordings.

Published
2024
Full Text
View/download PDF

21. Evolution of Einkorn wheat centromeres is driven by the mutualistic interplay of two LTR retrotransposons

Author

Matthias Heuberger, Dal-Hoe Koo, Hanin Ibrahim Ahmed, Vijay K. Tiwari, Michael Abrouk, Jesse Poland, Simon G. Krattinger, and Thomas Wicker

Subjects
Centromere evolution, Transposable element population genetics, Centromere stability, Genetics, QH426-470
Abstract

Abstract Background Centromere function is highly conserved across eukaryotes, but the underlying centromeric DNA sequences vary dramatically between species. Centromeres often contain a high proportion of repetitive DNA, such as tandem repeats and/or transposable elements (TEs). Einkorn wheat centromeres lack tandem repeat arrays and are instead composed mostly of the two long terminal repeat (LTR) retrotransposon families RLG_Cereba and RLG_Quinta which specifically insert in centromeres. However, it is poorly understood how these two TE families relate to each other and if and how they contribute to centromere function and evolution. Results Based on conservation of diagnostic motifs (LTRs, integrase and primer binding site and polypurine-tract), we propose that RLG_Cereba and RLG_Quinta are a pair of autonomous and non-autonomous partners, in which the autonomous RLG_Cereba contributes all the proteins required for transposition, while the non-autonomous RLG_Quinta contributes GAG protein. Phylogenetic analysis of predicted GAG proteins showed that the RLG_Cereba lineage was present for at least 100 million years in monocotyledon plants. In contrast, RLG_Quinta evolved from RLG_Cereba between 28 and 35 million years ago in the common ancestor of oat and wheat. Interestingly, the integrase of RLG_Cereba is fused to a so-called CR-domain, which is hypothesized to guide the integrase to the functional centromere. Indeed, ChIP-seq data and TE population analysis show only the youngest subfamilies of RLG_Cereba and RLG_Quinta are found in the active centromeres. Importantly, the LTRs of RLG_Quinta and RLG_Cereba are strongly associated with the presence of the centromere-specific CENH3 histone variant. We hypothesize that the LTRs of RLG_Cereba and RLG_Quinta contribute to wheat centromere integrity by phasing and/or placing CENH3 nucleosomes, thus favoring their persistence in the competitive centromere-niche. Conclusion Our data show that RLG_Cereba cross-mobilizes the non-autonomous RLG_Quinta retrotransposons. New copies of both families are specifically integrated into functional centromeres presumably through direct binding of the integrase CR domain to CENH3 histone variants. The LTRs of newly inserted RLG_Cereba and RLG_Quinta elements, in turn, recruit and/or phase new CENH3 deposition. This mutualistic interplay between the two TE families and the plant host dynamically maintains wheat centromeres.

Published
2024
Full Text
View/download PDF

22. Using Digital Technology in the Classroom Adds New Professional Tasks for (Student) Teachers

Author

Stéphane Colognesi and Vanessa Hanin

Abstract

This article discusses a study conducted among final-year undergraduate students preparing to become primary school teachers. These students participated in an eight-week module on digital technology, working in pairs to create, test and analyse pedagogical scenarios using and not using digital tablets. The study aimed to address two main questions: (1) How do the views of pre-service teachers change as a result of a technology instruction programme? (2) What new professional tasks emerge when they use digital technologies in their internship teaching practices? Various data sources were collected and analysed. The findings revealed that none of the pre-service teachers held entirely negative views on digital technologies after the module. They became more comfortable using them in their classrooms, especially for teaching oral language, while for mathematics they found them less useful. Furthermore, the results showed that eight new professional tasks appeared when teaching with digital technologies.

Published
2024
Full Text
View/download PDF

24. Quantitative CLTs in Deep Neural Networks

Author

Favaro, Stefano, Hanin, Boris, Marinucci, Domenico, Nourdin, Ivan, and Peccati, Giovanni

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Mathematics - Probability, Statistics - Machine Learning
Abstract

We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant $n$. Under mild assumptions on the non-linearity, we obtain quantitative bounds on normal approximations valid at large but finite $n$ and any fixed network depth. Our theorems show both for the finite-dimensional distributions and the entire process, that the distance between a random fully connected network (and its derivatives) to the corresponding infinite width Gaussian process scales like $n^{-\gamma}$ for $\gamma>0$, with the exponent depending on the metric used to measure discrepancy. Our bounds are strictly stronger in terms of their dependence on network width than any previously available in the literature; in the one-dimensional case, we also prove that they are optimal, i.e., we establish matching lower bounds.

Published
2023

25. Principles for Initialization and Architecture Selection in Graph Neural Networks with ReLU Activations

Author

DeZoort, Gage and Hanin, Boris

Subjects
Statistics - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Machine Learning, High Energy Physics - Experiment, Mathematics - Probability
Abstract

This article derives and validates three principles for initialization and architecture selection in finite width graph neural networks (GNNs) with ReLU activations. First, we theoretically derive what is essentially the unique generalization to ReLU GNNs of the well-known He-initialization. Our initialization scheme guarantees that the average scale of network outputs and gradients remains order one at initialization. Second, we prove in finite width vanilla ReLU GNNs that oversmoothing is unavoidable at large depth when using fixed aggregation operator, regardless of initialization. We then prove that using residual aggregation operators, obtained by interpolating a fixed aggregation operator with the identity, provably alleviates oversmoothing at initialization. Finally, we show that the common practice of using residual connections with a fixup-type initialization provably avoids correlation collapse in final layer features at initialization. Through ablation studies we find that using the correct initialization, residual aggregation operators, and residual connections in the forward pass significantly and reliably speeds up early training dynamics in deep ReLU GNNs on a variety of tasks., Comment: Comments appreciated

Published
2023

26. Depth Dependence of $\mu$P Learning Rates in ReLU MLPs

Author

Jelassi, Samy, Hanin, Boris, Ji, Ziwei, Reddi, Sashank J., Bhojanapalli, Srinadh, and Kumar, Sanjiv

Subjects
Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

In this short note we consider random fully connected ReLU networks of width $n$ and depth $L$ equipped with a mean-field weight initialization. Our purpose is to study the dependence on $n$ and $L$ of the maximal update ($\mu$P) learning rate, the largest learning rate for which the mean squared change in pre-activations after one step of gradient descent remains uniformly bounded at large $n,L$. As in prior work on $\mu$P of Yang et. al., we find that this maximal update learning rate is independent of $n$ for all but the first and last layer weights. However, we find that it has a non-trivial dependence of $L$, scaling like $L^{-3/2}.$

Published
2023

37. Caffeine intake and its association with nutrition, sleep, and physical activity among schoolchildren in the United Arab Emirates: a national cross-sectional study

Author

Faris, MoezAlIslam E., Saif, Eman Rashid, Turki, Eman Ali, Abdelrahim, Dana N., Abu-Qiyas, Salma, Shihab, Katia Abi, Zeb, Falak, Hasan, Haydar, Hashim, Mona S., Radwan, Hadia M., Naja, Farah, Ismail, Leila Cheikh, Osaili, Tareq M., Kassem, Hanin, Al Rajaby, Radhiya, Obaideen, Khaled, and Obaid, Reyad Shaker

Published
2024
Full Text
View/download PDF

40. Improving consumer understanding of pesticide toxicity labels: experimental evidence

Author

Hanin Hosni, Michelle Segovia, Shuoli Zhao, Marco A. Palma, and Theodoros Skevas

Subjects
Medicine, Science
Abstract

Abstract Consumers often inadvertently misperceive the health hazards associated with over-the-counter pesticides under the current textual labeling policy, potentially leading to improper use. We conducted an incentivized framed field experiment with eye tracking to evaluate the effectiveness of the current pesticide labels that convey risk using signal words (Caution, Warning, Danger) compared to two visually focused label alternatives: traffic light colors and skull intensity symbols. A total of 166 participants were randomly assigned to one of three label formats and asked to rank toxicity levels and make purchasing decisions within multiple price lists. Results show that signal words fail to adequately communicate toxicity levels. Specifically, participants’ correct assessment of toxicity level dramatically improves from 54% under the existing signal word label to 95% under the traffic light and 83% under the skull intensity symbol labels. We also find that participants are more likely to choose the less toxic alternatives under the new labels, suggesting the current labeling system may affect choice and have unintended adverse effects on human health.

Published
2024
Full Text
View/download PDF

41. Controllable Skyrmionic Phase Transition between Néel Skyrmions and Bloch Skyrmionic Bubbles in van der Waals Ferromagnet Fe3‐δGeTe2

Author

Liu, Chen, Jiang, Jiawei, Zhang, Chenhui, Wang, Qingping, Zhang, Huai, Zheng, Dongxing, Li, Yan, Ma, Yinchang, Algaidi, Hanin, Gao, Xingsen, Hou, Zhipeng, Mi, Wenbo, Liu, Jun‐ming, Qiu, Ziqiang, and Zhang, Xixiang

Subjects
Quantum Physics, Physical Sciences, 2D ferromagnet Fe3-& delta, GeTe2, Dzyaloshinskii-Moriya interaction, Fe atom displacement, magnetic skyrmions, 2D ferromagnet Fe3-δGeTe2
Abstract

The van der Waals (vdW) ferromagnet Fe3-δ GeTe2 has garnered significant research interest as a platform for skyrmionic spin configurations, that is, skyrmions and skyrmionic bubbles. However, despite extensive efforts, the origin of the Dzyaloshinskii-Moriya interaction (DMI) in Fe3-δ GeTe2 remains elusive, making it challenging to acquire these skyrmionic phases in a controlled manner. In this study, it is demonstrated that the Fe content in Fe3-δ GeTe2 has a profound effect on the crystal structure, DMI, and skyrmionic phase. For the first time, a marked increase in Fe atom displacement with decreasing Fe content is observed, transforming the original centrosymmetric crystal structure into a non-centrosymmetric symmetry, leading to a considerable DMI. Additionally, by varying the Fe content and sample thickness, a controllable transition between Néel-type skyrmions and Bloch-type skyrmionic bubbles is achieved, governed by a delicate interplay between dipole-dipole interaction and the DMI. The findings offer novel insights into the variable skyrmionic phases in Fe3-δ GeTe2 and provide the impetus for developing vdW ferromagnet-based spintronic devices.

Published
2023

42. Review and standard operating procedures for collection of biospecimens and analysis of biomarkers in new onset refractory status epilepticus.

Author

Hanin, Aurélie, Cespedes, Jorge, Pulluru, Yashwanth, Gopaul, Margaret, Aronica, Eleonora, Decampo, Danielle, Helbig, Ingo, Howe, Charles, Huttner, Anita, Koh, Sookyong, Navarro, Vincent, Taraschenko, Olga, Vezzani, Annamaria, Wilson, Michael, Xian, Julie, Gaspard, Nicolas, and Hirsch, Lawrence

Subjects
biomarkers, biospecimens, new onset refractory status epilepticus, standard operating procedures, Humans, Biological Specimen Banks, Status Epilepticus, Seizures, Drug Resistant Epilepsy, Encephalitis, Biomarkers
Abstract

New onset refractory status epilepticus (NORSE), including its subtype with a preceding febrile illness known as febrile infection-related epilepsy syndrome (FIRES), is one of the most severe forms of status epilepticus. The exact causes of NORSE are currently unknown, and there is so far no disease-specific therapy. Identifying the underlying pathophysiology and discovering specific biomarkers, whether immunologic, infectious, genetic, or other, may help physicians in the management of patients with NORSE. A broad spectrum of biomarkers has been proposed for status epilepticus patients, some of which were evaluated for patients with NORSE. Nonetheless, none has been validated, due to significant variabilities in study cohorts, collected biospecimens, applied analytical methods, and defined outcome endpoints, and to small sample sizes. The NORSE Institute established an open NORSE/FIRES biorepository for health-related data and biological samples allowing the collection of biospecimens worldwide, promoting multicenter research and sharing of data and specimens. Here, we suggest standard operating procedures for biospecimen collection and biobanking in this rare condition. We also propose criteria for the appropriate use of previously collected biospecimens. We predict that the widespread use of standardized procedures will reduce heterogeneity, facilitate the future identification of validated biomarkers for NORSE, and provide a better understanding of the pathophysiology and best clinical management for these patients.

Published
2023

43. Bayesian Interpolation with Deep Linear Networks

Author

Hanin, Boris and Zlokapa, Alexander

Subjects
Statistics - Machine Learning, Computer Science - Machine Learning, Mathematics - Probability
Abstract

Characterizing how neural network depth, width, and dataset size jointly impact model quality is a central problem in deep learning theory. We give here a complete solution in the special case of linear networks with output dimension one trained using zero noise Bayesian inference with Gaussian weight priors and mean squared error as a negative log-likelihood. For any training dataset, network depth, and hidden layer widths, we find non-asymptotic expressions for the predictive posterior and Bayesian model evidence in terms of Meijer-G functions, a class of meromorphic special functions of a single complex variable. Through novel asymptotic expansions of these Meijer-G functions, a rich new picture of the joint role of depth, width, and dataset size emerges. We show that linear networks make provably optimal predictions at infinite depth: the posterior of infinitely deep linear networks with data-agnostic priors is the same as that of shallow networks with evidence-maximizing data-dependent priors. This yields a principled reason to prefer deeper networks when priors are forced to be data-agnostic. Moreover, we show that with data-agnostic priors, Bayesian model evidence in wide linear networks is maximized at infinite depth, elucidating the salutary role of increased depth for model selection. Underpinning our results is a novel emergent notion of effective depth, given by the number of hidden layers times the number of data points divided by the network width; this determines the structure of the posterior in the large-data limit.

Published
2022

44. Maximal Initial Learning Rates in Deep ReLU Networks

Author

Iyer, Gaurav, Hanin, Boris, and Rolnick, David

Subjects
Statistics - Machine Learning, Computer Science - Machine Learning
Abstract

Training a neural network requires choosing a suitable learning rate, which involves a trade-off between speed and effectiveness of convergence. While there has been considerable theoretical and empirical analysis of how large the learning rate can be, most prior work focuses only on late-stage training. In this work, we introduce the maximal initial learning rate $\eta^{\ast}$ - the largest learning rate at which a randomly initialized neural network can successfully begin training and achieve (at least) a given threshold accuracy. Using a simple approach to estimate $\eta^{\ast}$, we observe that in constant-width fully-connected ReLU networks, $\eta^{\ast}$ behaves differently from the maximum learning rate later in training. Specifically, we find that $\eta^{\ast}$ is well predicted as a power of depth $\times$ width, provided that (i) the width of the network is sufficiently large compared to the depth, and (ii) the input layer is trained at a relatively small learning rate. We further analyze the relationship between $\eta^{\ast}$ and the sharpness $\lambda_{1}$ of the network at initialization, indicating they are closely though not inversely related. We formally prove bounds for $\lambda_{1}$ in terms of depth $\times$ width that align with our empirical results., Comment: International Conference on Machine Learning (ICML) 2023

Published
2022

45. Smoking Behaviors, Dietary Patterns, and Dyspepsia among Indonesian Adults in 2022

Author

Dwi Aprilawati, Gerald Sebastian Davis, Muftihana Hanin Nuha, Zsa Zsa Ollyvia, Aulia Nur Fadilla, Ananda Rahmadanti, Nadhifa Tanesha Aufazhafarin, Nabila Ananda Kloping, Pamarga Priyambodo, Andreas Novaldi Watang, Nandiwardhana Dhira Pranaya Lumaksono, and Sirazul Munir

Subjects
dyspepsia, tobacco use, dietary pattern, dyspepsia risk factors, Medicine
Abstract

Background: The global prevalence of dyspeptic syndrome among adults was estimated to be 1.8–57% in 2014, with an average of 20.8%. In a pilot survey involving 30 respondents, we found 21 acute respiratory infections, 16 dyspeptic syndromes, and 6 hypertension cases. Smoking was observed in 70% of the total respondents. This was the basis for conducting additional research in 2022 to examine the relationship between dyspepsia, smoking behaviors, and dietary patterns among adults in Krajan Hamlet, Malang, Indonesia. Methods: This quantitative study used an analytical observational design with a cross-sectional approach. The sample consisted of 70 adults aged 18–65, selected by random sampling. The collected data were analyzed using the Chi-squared test (p2 years and smoked >10 cigarettes daily. A total of 11.4% of subjects exhibited eating habits of consuming trigger foods that put them at a high risk for dyspepsia. Specifically, 42.0% of subjects had an eating interval of >6 hours, while 87.1% had a meal frequency that led to a high risk of developing dyspepsia. There was a significant relationship between a high-risk diet and dyspeptic syndrome (p=0.037). Conclusion: High-risk diets exhibited a relationship with the prevalence of dyspeptic syndrome. Early detection is crucial in an effort to prevent and reduce the incidence and complications of dyspeptic syndrome. Highlights: • This study investigated the relationship between dyspepsia, smoking behaviors, and dietary patterns, which are often overlooked, particularly among those living in rural areas. • The findings of this study are anticipated to raise awareness regarding the detrimental impacts of smoking and unhealthy diets on dyspepsia.

Published
2024
Full Text
View/download PDF

46. Seed quality as a proxy of climate-ready orphan legumes: the need for a multidisciplinary and multi-actor vision

Author

Alma Balestrazzi, Cinzia Calvio, Anca Macovei, Andrea Pagano, Patrick Laux, Hassane Moutahir, Loїc Rajjou, Eleni Tani, Dimosthenis Chachalis, Christos Katsis, Lamiae Ghaouti, Said Gmouh, Sanaa Majid, Amine Elleuch, Moez Hanin, Bassem Khemakhem, Hanen El Abed, Joao Nunes, Susana Araújo, Aziz Benhamrouche, and Mohand Bersi

Subjects
climate change, Greater Mediterranean Region, resilience, food security, multi-actor approach, underutilized legumes, Plant culture, SB1-1110
Abstract

In developing countries, orphan legumes stand at the forefront in the struggle against climate change. Their high nutrient value is crucial in malnutrition and chronic diseases prevention. However, as the ‘orphan’ definition suggests, their seed systems are still underestimated and seed production is scanty. Seed priming is an effective, sustainable strategy to boost seed quality in orphan legumes for which up-to-date guidelines are required to guarantee reliable and reproducible results. How far are we along this path? What do we expect from seed priming? This brings to other relevant questions. What is the socio-economic relevance of orphan legumes in the Mediterranean Basin? How to potentiate a broader cultivation in specific regions? The case study of the BENEFIT-Med (Boosting technologies of orphan legumes towards resilient farming systems) project, developed by multidisciplinary research networks, envisions a roadmap for producing new knowledge and innovative technologies to improve seed productivity through priming, with the long-term objective of promoting sustainability and food security for/in the climate-sensitive regions. This review highlights the existing drawbacks that must be overcome before orphan legumes could reach the state of ‘climate-ready crops’. Only by the integration of knowledge in seed biology, technology and agronomy, the barrier existing between research bench and local agricultural fields may be overcome, generating high-impact technical innovations for orphan legumes. We intend to provide a powerful message to encourage future research in line with the United Nations Agenda 2030 for Sustainable Development.

Published
2024
Full Text
View/download PDF

50. A Machine Learning Approach for Running Grade Classification Using IMUs Data

Author

Chaker, Abdelbadia, Atiga, Hanin, Donahue, Seth, Robinson, Rachel, Chebbi, Aida, Bennour, Sami, Hahn, Mike, Ceccarelli, Marco, Series Editor, Corves, Burkhard, Advisory Editor, Glazunov, Victor, Advisory Editor, Hernández, Alfonso, Advisory Editor, Huang, Tian, Advisory Editor, Jauregui Correa, Juan Carlos, Advisory Editor, Takeda, Yukio, Advisory Editor, Agrawal, Sunil K., Advisory Editor, Romdhane, Lotfi, editor, Mlika, Abdelfattah, editor, Zeghloul, Saïd, editor, Chaker, Abdelbadia, editor, and Laribi, Med Amine, editor

Published
2024
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results