Search

Teachers' professional identities are the foundation of their practice. Previous scholarship has largely overlooked the extent to which the broader reform culture shapes teachers' professional identities. In this study, I draw on survey data from 950 teachers across four US states (California, New York, Florida, and Texas) to examine the extent to which teachers' professional identities are associated with what I term "institutionalized conceptions" of their roles. Across diverse state policy contexts, I find that teachers draw upon a shared set of institutionalized conceptions of their roles, which are associated with their professional identities. The findings suggest that the taken-for-granted ways society frames teaching may be associated with dimensions of teachers' professional identity, such as self-efficacy and professional commitment.

We study two strange phenomena in auto-regressive Transformers: (1) the dominance of the first token in attention heads; (2) the occurrence of large outlier activations in the hidden states. We find that popular large language models, such as Llama attend maximally to the first token in 98% of attention heads, a behaviour we attribute to the softmax function. To mitigate this issue, we propose a reformulation of softmax to softmax-1. Furthermore, we identify adaptive optimisers, e.g. Adam, as the primary contributor to the large outlier activations and introduce OrthoAdam, a novel optimiser that utilises orthogonal matrices to transform gradients, to address this issue. Finally, not only do our methods prevent these phenomena from occurring, but additionally, they enable Transformers to sustain their performance when quantised using basic algorithms, something that standard methods are unable to do. In summary, our methods reduce the attention proportion on the first token from 65% to 3.3%, the activation kurtosis in the hidden states from 1657 to 3.1, and perplexity penalty under 4-bit weight quantisation from 3565 to 0.3., Comment: 10 pages

We present a novel Hand-pose Embedding Interactive System (HpEIS) as a virtual sensor, which maps users' flexible hand poses to a two-dimensional visual space using a Variational Autoencoder (VAE) trained on a variety of hand poses. HpEIS enables visually interpretable and guidable support for user explorations in multimedia collections, using only a camera as an external hand pose acquisition device. We identify general usability issues associated with system stability and smoothing requirements through pilot experiments with expert and inexperienced users. We then design stability and smoothing improvements, including hand-pose data augmentation, an anti-jitter regularisation term added to loss function, stabilising post-processing for movement turning points and smoothing post-processing based on One Euro Filters. In target selection experiments (n=12), we evaluate HpEIS by measures of task completion time and the final distance to target points, with and without the gesture guidance window condition. Experimental responses indicate that HpEIS provides users with a learnable, flexible, stable and smooth mid-air hand movement interaction experience., Comment: 6 pages, 8 figures, 3 tables

Brain-body-computer interfaces aim to provide a fluid and natural way for humans to interact with technology. Among noninvasive interfaces, surface electromyogram (sEMG) signals have shown particular utility. However, much remains unknown about how sEMG is affected by various physiological and anatomical factors and how these confounds might affect gesture decoding across individuals or groups. In this article, we show that sEMG signals evince non-Euclidean graph data structure that is defined by a set of orthogonal axes and explain the signal distribution shift across individuals. We provide a dataset of upper limb sEMG signals and physiological measures of 91 adults as they perform 10 different hand gestures. Participants were selected to be representative of various age groups (18to 92 years) and BMI (healthy, overweight, and obese). Additional anatomical or physiological measures that might impact sEMG signals were also collected, such as skin hydration and elasticity. The article describes the inherent structure of sEMG data and provides methods to construct differentiable signal features that can be used with machine learning algorithms that use backpropagation. We then analyze how those parameters correlate with various physiological measures to probe if they can induce bias against (or towards) certain population groups. We find that higher frequencies in sEMG, although comprising less power than lower ones, provide better gesture decoding and show less bias with regard to demographic, circumstantial, and physiological confounds (such as age, skin hydration, and skin elasticity)., Comment: 24 pages

The classic enumerative functions for counting colorings of a graph $G$, such as the chromatic polynomial $P(G,k)$, do so under the assumption that the given graph is labeled. In 1985, Hanlon defined and studied the chromatic polynomial for an unlabeled graph $\mathcal{G}$, $P(\mathcal{G}, k)$. Determining $P(\mathcal{G}, k)$ amounts to counting colorings under the action of automorphisms of $\mathcal{G}$. In this paper, we consider the problem of counting list colorings of unlabeled graphs. List coloring of graphs is a widely studied generalization of classic coloring that was introduced by Vizing and by Erd\H{o}s, Rubin, and Taylor in the 1970s. In 1990, Kostochka and Sidorenko introduced the list color function $P_\ell(G,k)$ which is the guaranteed number of list colorings of a labeled graph $G$ over all $k$-list assignments of $G$. In this paper, we extend Hanlon's definition to the list context and define the unlabeled list color function, $P_\ell(\mathcal{G}, k)$, of an unlabeled graph $\mathcal{G}$. In this context, we pursue a fundamental question whose analogues have driven much of the research on counting list colorings and its generalizations: For a given unlabeled graph $\mathcal{G}$, does $P_\ell(\mathcal{G}, k) = P(\mathcal{G}, k)$ when $k$ is large enough? We show the answer to this question is yes for a large class of unlabeled graphs that include point-determining graphs (also known as irreducible graphs and as mating graphs)., Comment: 14 pages

We study the fundamental scheduling problem $1\mid r_j\mid\sum w_j U_j$: schedule a set of $n$ jobs with weights, processing times, release dates, and due dates on a single machine, such that each job starts after its release date and we maximize the weighted number of jobs that complete execution before their due date. Problem $1\mid r_j\mid\sum w_j U_j$ generalizes both Knapsack and Partition, and the simplified setting without release dates was studied by Hermelin et al. [Annals of Operations Research, 2021] from a parameterized complexity viewpoint. Our main contribution is a thorough complexity analysis of $1\mid r_j\mid\sum w_j U_j$ in terms of four key problem parameters: the number $p_\#$ of processing times, the number $w_\#$ of weights, the number $d_\#$ of due dates, and the number $r_\#$ of release dates of the jobs. $1\mid r_j\mid\sum w_j U_j$ is known to be weakly para-NP-hard even if $w_\#+d_\#+r_\#$ is constant, and Heeger and Hermelin [ESA, 2024] recently showed (weak) W[1]-hardness parameterized by $p_\#$ or $w_\#$ even if $r_\#$ is constant. Algorithmically, we show that $1\mid r_j\mid\sum w_j U_j$ is fixed-parameter tractable parameterized by $p_\#$ combined with any two of the remaining three parameters $w_\#$, $d_\#$, and $r_\#$. We further provide pseudo-polynomial XP-time algorithms for parameter $r_\#$ and $d_\#$. To complement these algorithms, we show that $1\mid r_j\mid\sum w_j U_j$ is (strongly) W[1]-hard when parameterized by $d_\#+r_\#$ even if $w_\#$ is constant. Our results provide a nearly complete picture of the complexity of $1\mid r_j\mid\sum w_j U_j$ for $p_\#$, $w_\#$, $d_\#$, and $r_\#$ as parameters, and extend those of Hermelin et al. [Annals of Operations Research, 2021] for the problem $1\mid\mid\sum w_j U_j$ without release dates.

In extremal combinatorics, it is common to focus on structures that are minimal with respect to a certain property. In particular, critical and list-critical graphs occupy a prominent place in graph coloring theory. Stiebitz, Tuza, and Voigt introduced strongly critical graphs, i.e., graphs that are $k$-critical yet $L$-colorable with respect to every non-constant assignment $L$ of lists of size $k-1$. Here we strengthen this notion and extend it to the framework of DP-coloring (or correspondence coloring) by defining robustly $k$-critical graphs as those that are not $(k-1)$-DP-colorable, but only due to the fact that $\chi(G) = k$. We then seek general methods for constructing robustly critical graphs. Our main result is that if $G$ is a critical graph (with respect to ordinary coloring), then the join of $G$ with a sufficiently large clique is robustly critical; this is new even for strong criticality., Comment: 15 pages, 2 figures

In the field of detection and ranging, multiple complementary sensing modalities may be used to enrich the information obtained from a dynamic scene. One application of this sensor fusion is in public security and surveillance, whose efficacy and privacy protection measures must be continually evaluated. We present a novel deployment of sensor fusion for the discrete detection of concealed metal objects on persons whilst preserving their privacy. This is achieved by coupling off-the-shelf mmWave radar and depth camera technology with a novel neural network architecture that processes the radar signals using convolutional Long Short-term Memory (LSTM) blocks and the depth signal, using convolutional operations. The combined latent features are then magnified using a deep feature magnification to learn cross-modality dependencies in the data. We further propose a decoder, based on the feature extraction and embedding block, to learn an efficient upsampling of the latent space to learn the location of the concealed object in the spatial domain through radar feature guidance. We demonstrate the detection of presence and inference of 3D location of concealed metal objects with an accuracy of up to 95%, using a technique that is robust to multiple persons. This work provides a demonstration of the potential for cost effective and portable sensor fusion, with strong opportunities for further development.

[See paper for full abstract] Meta-analysis is a crucial tool for answering scientific questions. It is usually conducted on a relatively small amount of ``trusted'' data -- ideally from randomized, controlled trials -- which allow causal effects to be reliably estimated with minimal assumptions. We show how to answer causal questions much more precisely by making two changes. First, we incorporate untrusted data drawn from large observational databases, related scientific literature and practical experience -- without sacrificing rigor or introducing strong assumptions. Second, we train richer models capable of handling heterogeneous trials, addressing a long-standing challenge in meta-analysis. Our approach is based on conformal prediction, which fundamentally produces rigorous prediction intervals, but doesn't handle indirect observations: in meta-analysis, we observe only noisy effects due to the limited number of participants in each trial. To handle noise, we develop a simple, efficient version of fully-conformal kernel ridge regression, based on a novel condition called idiocentricity. We introduce noise-correcting terms in the residuals and analyze their interaction with a ``variance shaving'' technique. In multiple experiments on healthcare datasets, our algorithms deliver tighter, sounder intervals than traditional ones. This paper charts a new course for meta-analysis and evidence-based medicine, where heterogeneity and untrusted data are embraced for more nuanced and precise predictions., Comment: Full-length version of conference submission

We study the transition between N\'eel and columnar valence-bond solid ordering in two-dimensional $S=3/2$ square lattice quantum antiferromagnets with SO(3) symmetry. According to the deconfined criticality scenario, this transition can be direct and continuous like the well-studied $S=1/2$ case. To study the global phase diagram, we work with four multi-spin couplings with full rotational symmetry, that are free of the sign-problem of quantum Monte Carlo. Exploring the phase diagram with quantum Monte Carlo simulations, we find that the phase transition between N\'eel and valence-bond solid is strongly first-order in the parts of the phase diagram that we have accessed., Comment: 11 pages, 16 figures

Self-supervised learning (SSL) with vision transformers (ViTs) has proven effective for representation learning as demonstrated by the impressive performance on various downstream tasks. Despite these successes, existing ViT-based SSL architectures do not fully exploit the ViT backbone, particularly the patch tokens of the ViT. In this paper, we introduce a novel Semantic Graph Consistency (SGC) module to regularize ViT-based SSL methods and leverage patch tokens effectively. We reconceptualize images as graphs, with image patches as nodes and infuse relational inductive biases by explicit message passing using Graph Neural Networks into the SSL framework. Our SGC loss acts as a regularizer, leveraging the underexploited patch tokens of ViTs to construct a graph and enforcing consistency between graph features across multiple views of an image. Extensive experiments on various datasets including ImageNet, RESISC and Food-101 show that our approach significantly improves the quality of learned representations, resulting in a 5-10\% increase in performance when limited labeled data is used for linear evaluation. These experiments coupled with a comprehensive set of ablations demonstrate the promise of our approach in various settings.

In this work, we introduce Pixelsmith, a zero-shot text-to-image generative framework to sample images at higher resolutions with a single GPU. We are the first to show that it is possible to scale the output of a pre-trained diffusion model by a factor of 1000, opening the road for gigapixel image generation at no additional cost. Our cascading method uses the image generated at the lowest resolution as a baseline to sample at higher resolutions. For the guidance, we introduce the Slider, a tunable mechanism that fuses the overall structure contained in the first-generated image with enhanced fine details. At each inference step, we denoise patches rather than the entire latent space, minimizing memory demands such that a single GPU can handle the process, regardless of the image's resolution. Our experimental results show that Pixelsmith not only achieves higher quality and diversity compared to existing techniques, but also reduces sampling time and artifacts. The code for our work is available at https://github.com/Thanos-DB/Pixelsmith.

Given unstructured text, Large Language Models (LLMs) are adept at answering simple (single-hop) questions. However, as the complexity of the questions increase, the performance of LLMs degrade. We believe this is due to the overhead associated with understanding the complex question followed by filtering and aggregating unstructured information in the raw text. Recent methods try to reduce this burden by integrating structured knowledge triples into the raw text, aiming to provide a structured overview that simplifies information processing. However, this simplistic approach is query-agnostic and the extracted facts are ambiguous as they lack context. To address these drawbacks and to enable LLMs to answer complex (multi-hop) questions with ease, we propose to use a knowledge graph (KG) that is context-aware and is distilled to contain query-relevant information. The use of our compressed distilled KG as input to the LLM results in our method utilizing up to $67\%$ fewer tokens to represent the query relevant information present in the supporting documents, compared to the state-of-the-art (SoTA) method. Our experiments show consistent improvements over the SoTA across several metrics (EM, F1, BERTScore, and Human Eval) on two popular benchmark datasets (HotpotQA and MuSiQue)., Comment: Accepted at ACL 2024 in the main track

We introduce a simple lattice spin model that is written in terms of the well-known four-dimensional $\gamma$-matrix representation of the Clifford algebra. The local spins with a four-dimensional Hilbert space transform in a spinorial $(1/2,0) \oplus (0,1/2)$ representation of $SO(4)$, a symmetry of our model. When studied on a chain, and as a function of a transverse field tuning parameter, our model undergoes a quantum phase transition from a valence bond solid phase to a critical phase that is described by an $SU(2)_1$ WZW field theory., Comment: 12 pages, 9 figures; added references for section 2, corrected a typo in appendix D

Mitigating hallucinations in large vision-language models (LVLMs) remains an open problem. Recent benchmarks do not address hallucinations in open-ended free-form responses, which we term "Type I hallucinations". Instead, they focus on hallucinations responding to very specific question formats -- typically a multiple-choice response regarding a particular object or attribute -- which we term "Type II hallucinations". Additionally, such benchmarks often require external API calls to models which are subject to change. In practice, we observe that a reduction in Type II hallucinations does not lead to a reduction in Type I hallucinations but rather that the two forms of hallucinations are often anti-correlated. To address this, we propose THRONE, a novel object-based automatic framework for quantitatively evaluating Type I hallucinations in LVLM free-form outputs. We use public language models (LMs) to identify hallucinations in LVLM responses and compute informative metrics. By evaluating a large selection of recent LVLMs using public datasets, we show that an improvement in existing metrics do not lead to a reduction in Type I hallucinations, and that established benchmarks for measuring Type I hallucinations are incomplete. Finally, we provide a simple and effective data augmentation method to reduce Type I and Type II hallucinations as a strong baseline., Comment: In CVPR 2024

Despite substantial progress in machine learning for scientific discovery in recent years, truly de novo design of small molecules which exhibit a property of interest remains a significant challenge. We introduce LambdaZero, a generative active learning approach to search for synthesizable molecules. Powered by deep reinforcement learning, LambdaZero learns to search over the vast space of molecules to discover candidates with a desired property. We apply LambdaZero with molecular docking to design novel small molecules that inhibit the enzyme soluble Epoxide Hydrolase 2 (sEH), while enforcing constraints on synthesizability and drug-likeliness. LambdaZero provides an exponential speedup in terms of the number of calls to the expensive molecular docking oracle, and LambdaZero de novo designed molecules reach docking scores that would otherwise require the virtual screening of a hundred billion molecules. Importantly, LambdaZero discovers novel scaffolds of synthesizable, drug-like inhibitors for sEH. In in vitro experimental validation, a series of ligands from a generated quinazoline-based scaffold were synthesized, and the lead inhibitor N-(4,6-di(pyrrolidin-1-yl)quinazolin-2-yl)-N-methylbenzamide (UM0152893) displayed sub-micromolar enzyme inhibition of sEH.

This research paper explores ways to apply Federated Learning (FL) and Differential Privacy (DP) techniques to population-scale Electrocardiogram (ECG) data. The study learns a multi-label ECG classification model using FL and DP based on 1,565,849 ECG tracings from 7 hospitals in Alberta, Canada. The FL approach allowed collaborative model training without sharing raw data between hospitals while building robust ECG classification models for diagnosing various cardiac conditions. These accurate ECG classification models can facilitate the diagnoses while preserving patient confidentiality using FL and DP techniques. Our results show that the performance achieved using our implementation of the FL approach is comparable to that of the pooled approach, where the model is trained over the aggregating data from all hospitals. Furthermore, our findings suggest that hospitals with limited ECGs for training can benefit from adopting the FL model compared to single-site training. In addition, this study showcases the trade-off between model performance and data privacy by employing DP during model training. Our code is available at https://github.com/vikhyatt/Hospital-FL-DP., Comment: Accepted for ICMHI 2024

We propose a novel transformer-style architecture called Global-Local Filter Network (GLFNet) for medical image segmentation and demonstrate its state-of-the-art performance. We replace the self-attention mechanism with a combination of global-local filter blocks to optimize model efficiency. The global filters extract features from the whole feature map whereas the local filters are being adaptively created as 4x4 patches of the same feature map and add restricted scale information. In particular, the feature extraction takes place in the frequency domain rather than the commonly used spatial (image) domain to facilitate faster computations. The fusion of information from both spatial and frequency spaces creates an efficient model with regards to complexity, required data and performance. We test GLFNet on three benchmark datasets achieving state-of-the-art performance on all of them while being almost twice as efficient in terms of GFLOP operations.