Search

Human vision is capable of focusing on subtle visual cues at high resolution by relying on a foveal view coupled with an attention mechanism. Recently, there have been several studies that proposed deep reinforcement learning based attention models. However, these studies do not explicitly consider the design of a foveal representation and its effect on an attention system is unclear. In this paper, we investigate the effect of using a hierarchy of visual streams in training an efficient attention model towards achieving a human-level sharp vision. We perform our evaluation on a simulated human-robot interaction task where the agent attends to faces that are looking at it. The experimental results show that the performance of the system relies on factors such as the number of visual streams, their relative field-of-view and we demonstrate that maintaining a hierarchy within the visual streams is crucial to learn attention strategies., Workshop on Sixth International Conference on Learning Representations(ICLR 2018), May 3, 2018, Vancouver Canada

While Large Language Models (LLMs) have achieved remarkable performance in many tasks, much about their inner workings remains unclear. In this study, we present novel experimental insights into the resilience of LLMs, particularly GPT-4, when subjected to extensive character-level permutations. To investigate this, we first propose the Scrambled Bench, a suite designed to measure the capacity of LLMs to handle scrambled input, in terms of both recovering scrambled sentences and answering questions given scrambled context. The experimental results indicate that most powerful LLMs demonstrate the capability akin to typoglycemia, a phenomenon where humans can understand the meaning of words even when the letters within those words are scrambled, as long as the first and last letters remain in place. More surprisingly, we found that only GPT-4 nearly flawlessly processes inputs with unnatural errors, even under the extreme condition, a task that poses significant challenges for other LLMs and often even for humans. Specifically, GPT-4 can almost perfectly reconstruct the original sentences from scrambled ones, decreasing the edit distance by 95%, even when all letters within each word are entirely scrambled. It is counter-intuitive that LLMs can exhibit such resilience despite severe disruption to input tokenization caused by scrambled text., Comment: EMNLP 2023 (with an additional analysis section in appendix)

Learning from Demonstration (LfD) is a paradigm where humans demonstrate the procedure to perform complex tasks which can be used to train autonomous agents. However, the performance of LfD is highly sensitive to the quality of demonstrations which in turn depends on the user-interface. In this paper, we propose the use of Virtual Reality (VR) to develop an intuitive interface that enables users to provide good demonstrations. We apply this approach to the task of training a visual attention system which is a crucial component for tasks such as autonomous driving and human-robot interaction. We show that interaction time of few minutes is sufficient to train a deep neural network to successfully learn attention strategies.