Search

Self-supervised learning (SSL) is one strategy for addressing the paucity of labelled data in medical imaging by learning representations from unlabelled images. Contrastive and non-contrastive SSL methods produce learned representations that are similar for pairs of related images. Such pairs are commonly constructed by randomly distorting the same image twice. The videographic nature of ultrasound offers flexibility for defining the similarity relationship between pairs of images. In this study, we investigated the effect of utilizing proximal, distinct images from the same B-mode ultrasound video as pairs for SSL. Additionally, we introduced a sample weighting scheme that increases the weight of closer image pairs and demonstrated how it can be integrated into SSL objectives. Named Intra-Video Positive Pairs (IVPP), the method surpassed previous ultrasound-specific contrastive learning methods' average test accuracy on COVID-19 classification with the POCUS dataset by $\ge 1.3\%$. Detailed investigations of IVPP's hyperparameters revealed that some combinations of IVPP hyperparameters can lead to improved or worsened performance, depending on the downstream task. Guidelines for practitioners were synthesized based on the results, such as the merit of IVPP with task-specific hyperparameters, and the improved performance of contrastive methods for ultrasound compared to non-contrastive counterparts., Comment: 18 pages, 5 figures

The Latent Block Model (LBM) is a prominent model-based co-clustering method, returning parametric representations of each block cluster and allowing the use of well-grounded model selection methods. The LBM, while adapted in literature to handle different feature types, cannot be applied to datasets consisting of multiple disjoint sets of features, termed views, for a common set of observations. In this work, we introduce the multi-view LBM, extending the LBM method to multi-view data, where each view marginally follows an LBM. In the case of two views, the dependence between them is captured by a cluster membership matrix, and we aim to learn the structure of this matrix. We develop a likelihood-based approach in which parameter estimation uses a stochastic EM algorithm integrating a Gibbs sampler, and an ICL criterion is derived to determine the number of row and column clusters in each view. To motivate the application of multi-view methods, we extend recent work developing hypothesis tests for the null hypothesis that clusters of observations in each view are independent of each other. The testing procedure is integrated into the model estimation strategy. Furthermore, we introduce a penalty scheme to generate sparse row clusterings. We verify the performance of the developed algorithm using synthetic datasets, and provide guidance for optimal parameter selection. Finally, the multi-view co-clustering method is applied to a complex genomics dataset, and is shown to provide new insights for high-dimension multi-view problems.

As the complexity of logic designs increase, new avenues for testing digital hardware becomes necessary. Fuzz Testing (fuzzing) has recently received attention as a potential candidate for input vector generation on hardware designs. Using this technique, a fuzzer is used to generate an input to a logic design. Using a simulation engine, the logic design is given the generated stimulus and some metric of feedback is given to the fuzzer to aid in the input mutation. However, much like software fuzzing, hardware fuzzing uses code coverage as a metric to find new possible fuzzing paths. Unfortunately, as we show in this work, this coverage metric falls short of generic on some hardware designs where designers have taken a more direct approach at expressing a particular microarchitecture, or implementation, of the desired hardware. With this work, we introduce a new coverage metric which employs not code coverage, but state coverage internal to a design. By observing changes in signals within the logic circuit under testing, we are able to explore the state space of the design and provide feedback to a fuzzer engine for input generation. Our approach, Value-Guided Fuzzing (VGF), provides a generic metric of coverage which can be applied to any design regardless of its implementation. In this paper, we introduce our state-based VGF metric as well as a sample implementation which can be used with any VPI, DPI, VHPI, or FLI compliant simulator, making it completely HDL agnostic. We demonstrate the generality of VGF and show how our sample implementation is capable of finding bugs considerably faster than previous approaches., Comment: 20 pages, 7 figures, 7 tables

In this study, we investigated whether self-supervised pretraining could produce a neural network feature extractor applicable to multiple classification tasks in B-mode lung ultrasound analysis. When fine-tuning on three lung ultrasound tasks, pretrained models resulted in an improvement of the average across-task area under the receiver operating curve (AUC) by 0.032 and 0.061 on local and external test sets respectively. Compact nonlinear classifiers trained on features outputted by a single pretrained model did not improve performance across all tasks; however, they did reduce inference time by 49% compared to serial execution of separate fine-tuned models. When training using 1% of the available labels, pretrained models consistently outperformed fully supervised models, with a maximum observed test AUC increase of 0.396 for the task of view classification. Overall, the results indicate that self-supervised pretraining is useful for producing initial weights for lung ultrasound classifiers., Comment: 10 pages, 5 figures, submitted to IEEE Access

Self-supervised pretraining has been observed to be effective at improving feature representations for transfer learning, leveraging large amounts of unlabelled data. This review summarizes recent research into its usage in X-ray, computed tomography, magnetic resonance, and ultrasound imaging, concentrating on studies that compare self-supervised pretraining to fully supervised learning for diagnostic tasks such as classification and segmentation. The most pertinent finding is that self-supervised pretraining generally improves downstream task performance compared to full supervision, most prominently when unlabelled examples greatly outnumber labelled examples. Based on the aggregate evidence, recommendations are provided for practitioners considering using self-supervised learning. Motivated by limitations identified in current research, directions and practices for future study are suggested, such as integrating clinical knowledge with theoretically justified self-supervised learning methods, evaluating on public datasets, growing the modest body of evidence for ultrasound, and characterizing the impact of self-supervised pretraining on generalization., Comment: 32 pages, 6 figures, a literature survey submitted to BMC Medical Imaging

Self-supervised pretraining has been observed to improve performance in supervised learning tasks in medical imaging. This study investigates the utility of self-supervised pretraining prior to conducting supervised fine-tuning for the downstream task of lung sliding classification in M-mode lung ultrasound images. We propose a novel pairwise relationship that couples M-mode images constructed from the same B-mode image and investigate the utility of data augmentation procedure specific to M-mode lung ultrasound. The results indicate that self-supervised pretraining yields better performance than full supervision, most notably for feature extractors not initialized with ImageNet-pretrained weights. Moreover, we observe that including a vast volume of unlabelled data results in improved performance on external validation datasets, underscoring the value of self-supervision for improving generalizability in automatic ultrasound interpretation. To the authors' best knowledge, this study is the first to characterize the influence of self-supervised pretraining for M-mode ultrasound., Comment: 10 pages, 6 figures, to be published in the 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

We present our preliminary work on a multi-agent system involving the complex human phenomena of identity and dynamic teams. We outline our ongoing experimentation into understanding how these factors can eliminate some of the naive assumptions of current multi-agent approaches. These include a lack of complex heterogeneity between agents and unchanging team structures. We outline the human social psychological basis for identity, one's sense of self, and dynamic teams, the changing nature of human teams. We describe our application of these factors to a multi-agent system and our expectations for how they might improve the system's applicability to more complex problems, with specific relevance to ad hoc teamwork. We expect that the inclusion of more complex human processes, like identity and dynamic teams, will help with the eventual goal of having effective human-agent teams., Comment: This paper appeared at the Workshop on Ad Hoc Teamwork, at IJCAI 2022

We introduce a method of reversing the execution of imperative concurrent programs. Given an irreversible program, we describe the process of producing two versions. The first performs forward execution and saves information necessary for reversal. The second uses this saved information to simulate reversal. We propose using identifiers to overcome challenges of reversing concurrent programs. We prove this reversibility to be correct, showing that the initial program state is restored and that all saved information is used (garbage-free)., Comment: 48 pages, 21 figures

Any decision, such as one about who to hire, involves two components. First, a rational component, i.e., they have a good education, they speak clearly. Second, an affective component, based on observables such as visual features of race and gender, and possibly biased by stereotypes. Here we propose a method for measuring the affective, socially biased, component, thus enabling its removal. That is, given a decision-making process, these affective measurements remove the affective bias in the decision, rendering it fair across a set of categories defined by the method itself. We thus propose that this may solve three key problems in intersectional fairness: (1) the definition of categories over which fairness is a consideration; (2) an infinite regress into smaller and smaller groups; and (3) ensuring a fair distribution based on basic human rights or other prior information. The primary idea in this paper is that fairness biases can be measured using affective coherence, and that this can be used to normalize outcome mappings. We aim for this conceptual work to expose a novel method for handling fairness problems that uses emotional coherence as an independent measure of bias that goes beyond statistical parity.

One's ability to learn a generative model of the world without supervision depends on the extent to which one can construct abstract knowledge representations that generalize across experiences. To this end, capturing an accurate statistical structure from observational data provides useful inductive biases that can be transferred to novel environments. Here, we tackle the problem of learning to control dynamical systems by applying Bayesian nonparametric methods, which is applied to solve visual servoing tasks. This is accomplished by first learning a state space representation, then inferring environmental dynamics and improving the policies through imagined future trajectories. Bayesian nonparametric models provide automatic model adaptation, which not only combats underfitting and overfitting, but also allows the model's unbounded dimension to be both flexible and computationally tractable. By employing Gaussian processes to discover latent world dynamics, we mitigate common data efficiency issues observed in reinforcement learning and avoid introducing explicit model bias by describing the system's dynamics. Our algorithm jointly learns a world model and policy by optimizing a variational lower bound of a log-likelihood with respect to the expected free energy minimization objective function. Finally, we compare the performance of our model with the state-of-the-art alternatives for continuous control tasks in simulated environments.

This paper presents the design of a cooperative multi-player betting game, Trust-ya, as a model of some elements of status processes in human groups. The game is designed to elicit status-driven leader-follower behaviours as a means to observe and influence social hierarchy. It involves a Bach/Stravinsky game of deference in a group, in which people on each turn can either invest with another player or hope someone invests with them. Players who receive investment capital are able to gamble for payoffs from a central pool which then can be shared back with those who invested (but a portion of it may be kept, including all of it). The bigger gambles (people with more investors) get bigger payoffs. Thus, there is a natural tendency for players to coalesce as investors around a 'leader' who gambles, but who also shares sufficiently from their winnings to keep the investors 'hanging on'. The 'leader' will want to keep as much as possible for themselves, however. The game is played anonymously, but a set of 'status symbols' can be purchased which have no value in the game itself, but can serve as a 'cheap talk' communication device with other players. This paper introduces the game, relates it to status theory in social psychology, and shows some simple simulated and human experiments that demonstrate how the game can be used to study status processes and dynamics in human groups., Comment: 5 pages, 3 figures

Let $f(z) = \sum_{n=1}^\infty a_f(n)q^n$ be a holomorphic cuspidal newform with even integral weight $k\geq 2$, level $N$, trivial nebentypus, and no complex multiplication (CM). For all primes $p$, we may define $\theta_p\in [0,\pi]$ such that $a_f(p) = 2p^{(k-1)/2}\cos \theta_p$. The Sato-Tate conjecture states that the angles $\theta_p$ are equidistributed with respect to the probability measure $\mu_{\textrm{ST}}(I) = \frac{2}{\pi}\int_I \sin^2 \theta \; d\theta$, where $I\subseteq [0,\pi]$. Using recent results on the automorphy of symmetric-power $L$-functions due to Newton and Thorne, we explicitly bound the error term in the Sato-Tate conjecture when $f$ corresponds to an elliptic curve over $\mathbb{Q}$ of arbitrary conductor or when $f$ has squarefree level. In these cases, if $\pi_{f,I}(x) := \#\{ p \leq x : p \nmid N, \theta_p\in I\}$, and $\pi(x) := \# \{ p \leq x \}$, we prove the following bound: $$\left| \frac{\pi_{f,I}(x)}{\pi(x)} - \mu_{\textrm{ST}}(I)\right| \leq 58.1\frac{\log((k-1)N \log{x})}{\sqrt{\log{x}}} \qquad \text{for} \quad x \geq 3.$$ As an application, we give an explicit bound for the number of primes up to $x$ that violate the Atkin-Serre conjecture for $f$., Comment: 31 pages; minor improvements in exposition; minor typos fixed

In this paper the reversibility of executable Interval Temporal Logic (ITL) specifications is investigated. ITL allows for the reasoning about systems in terms of behaviours which are represented as non-empty sequences of states. It allows for the specification of systems at different levels of abstraction. At a high level this specification is in terms of properties, for instance safety and liveness properties. At concrete level one can specify a system in terms of programming constructs. One can execute these concrete specification, i.e., test and simulate the behaviour of the system. In this paper we will formalise this notion of executability of ITL specifications. ITL also has a reflection operator which allows for the reasoning about reversed behaviours. We will investigate the reversibility of executable ITL specifications, i.e., how one can use this reflection operator to reverse the concrete behaviour of a particular system., Comment: 11 pages, 0 figures, 1 table, abridged version to published in Reversible Computation 2021

Machine learning has the potential to aid in mitigating the human effects of climate change. Previous applications of machine learning to tackle the human effects in climate change include approaches like informing individuals of their carbon footprint and strategies to reduce it. For these methods to be the most effective they must consider relevant social-psychological factors for each individual. Of social-psychological factors at play in climate change, affect has been previously identified as a key element in perceptions and willingness to engage in mitigative behaviours. In this work, we propose an investigation into how affect could be incorporated to enhance machine learning based interventions for climate change. We propose using affective agent-based modelling for climate change as well as the use of a simulated climate change social dilemma to explore the potential benefits of affective machine learning interventions. Behavioural and informational interventions can be a powerful tool in helping humans adopt mitigative behaviours. We expect that utilizing affective ML can make interventions an even more powerful tool and help mitigative behaviours become widely adopted., Comment: Accepted paper at the Tackling Climate Change with Machine Learning workshop at NeurIPS 2020

The COVID-19 Pandemic has been described as the global challenge of our time, an enormous human tragedy with dramatic economic impacts. This paper describes the response and expected recovery process for Western Australia, where a rapid and effective response was implemented. This has enabled an early transition into an expected recovery both in health and economic terms. The positive lessons learned from this experience are documented as they emerge in order to support other states and nations as they address this issue globally in the near-term and consider enduring improvements for the longer term. While the authors have personal experience in the WA context, wider observations across Australia and selected international benchmarks are also included. Key lessons include the importance of good health advice in Australia's interest; timely, synchronized and aligned action at all levels of government; a program of well communicated, aligned health and economic measures which support all in society allowing a very high level of appropriate community behaviour, ensuring the health system was not overloaded; innovation in telehealth, testing, pandemic modelling, and integrated operations which also allowed essential industries to continue; and strong border and travel controls with highly effective isolation preventing community spread, ultimately enabling rapid elimination of the disease from the hospital system. In combination, these demonstrate that in the case of Western Australia the result of first eliminating the disease from the community, and then reopening the economy progressively at a strong pace, has enabled a world leading outcome in both in health and economic terms. The lessons from this experience are widely applicable, shareable both as supporting service to other regions and through knowledge transfer.

The Mordell-Weil groups $E(\mathbb{Q})$ of elliptic curves influence the structures of their quadratic twists $E_{-D}(\mathbb{Q})$ and the ideal class groups $\mathrm{CL}(-D)$ of imaginary quadratic fields. For appropriate $(u,v) \in \mathbb{Z}^2$, we define a family of homomorphisms $\Phi_{u,v}: E(\mathbb{Q}) \rightarrow \mathrm{CL}(-D)$ for particular negative fundamental discriminants $-D:=-D_E(u,v)$, which we use to simultaneously address questions related to lower bounds for class numbers, the structures of class groups, and ranks of quadratic twists. Specifically, given an elliptic curve $E$ of rank $r$, let $\Psi_E$ be the set of suitable fundamental discriminants $-D<0$ satisfying the following three conditions: the quadratic twist $E_{-D}$ has rank at least 1; $E_{\text{tor}}(\mathbb{Q})$ is a subgroup of $\mathrm{CL}(-D)$; and $h(-D)$ satisfies an effective lower bound which grows asymptotically like $c(E) \log (D)^{\frac{r}{2}}$ as $D \to \infty$. Then for any $\varepsilon > 0$, we show that as $X \to \infty$, we have $$\#\, \left\{-X < -D < 0: -D \in \Psi_E \right \} \, \gg_{\varepsilon} X^{\frac{1}{2}-\varepsilon}.$$ In particular, if $\ell \in \{3,5,7\}$ and $\ell \mid |E_{\mathrm{tor}}(\mathbb{Q})|$, then the number of such discriminants $-D$ for which $\ell \mid h(-D)$ is $\gg_{\varepsilon} X^{\frac{1}{2}-\varepsilon}.$ Moreover, assuming the Parity Conjecture, our results hold with the additional condition that the quadratic twist $E_{-D}$ has rank at least 2., Comment: 17 pages, 1 table

State-of-the-art neural dialogue systems excel at syntactic and semantic modelling of language, but often have a hard time establishing emotional alignment with the human interactant during a conversation. In this work, we bring Affect Control Theory (ACT), a socio-mathematical model of emotions for human-human interactions, to the neural dialogue generation setting. ACT makes predictions about how humans respond to emotional stimuli in social situations. Due to this property, ACT and its derivative probabilistic models have been successfully deployed in several applications of Human-Computer Interaction, including empathetic tutoring systems, assistive healthcare devices and two-person social dilemma games. We investigate how ACT can be used to develop affect-aware neural conversational agents, which produce emotionally aligned responses to prompts and take into consideration the affective identities of the interactants.