Search

Large language models (LLMs) equipped with chain-of-thoughts (CoT) prompting have shown significant multi-step reasoning capabilities in factual content like mathematics, commonsense, and logic. However, their performance in narrative reasoning, which demands greater abstraction capabilities, remains unexplored. This study utilizes tropes in movie synopses to assess the abstract reasoning abilities of state-of-the-art LLMs and uncovers their low performance. We introduce a trope-wise querying approach to address these challenges and boost the F1 score by 11.8 points. Moreover, while prior studies suggest that CoT enhances multi-step reasoning, this study shows CoT can cause hallucinations in narrative content, reducing GPT-4's performance. We also introduce an Adversarial Injection method to embed trope-related text tokens into movie synopses without explicit tropes, revealing CoT's heightened sensitivity to such injections. Our comprehensive analysis provides insights for future research directions., Comment: EMNLP 2024 Findings. The first two authors contributed equally. Code: https://github.com/Shelley1214/Trope

We study reward models for long-horizon manipulation tasks by learning from action-free videos and language instructions, which we term the visual-instruction correlation (VIC) problem. Recent advancements in cross-modality modeling have highlighted the potential of reward modeling through visual and language correlations. However, existing VIC methods face challenges in learning rewards for long-horizon tasks due to their lack of sub-stage awareness, difficulty in modeling task complexities, and inadequate object state estimation. To address these challenges, we introduce VICtoR, a novel hierarchical VIC reward model capable of providing effective reward signals for long-horizon manipulation tasks. VICtoR precisely assesses task progress at various levels through a novel stage detector and motion progress evaluator, offering insightful guidance for agents learning the task effectively. To validate the effectiveness of VICtoR, we conducted extensive experiments in both simulated and real-world environments. The results suggest that VICtoR outperformed the best existing VIC methods, achieving a 43% improvement in success rates for long-horizon tasks.

Emerging transportation technologies, such as ride-hailing and autonomous vehicles, are disrupting the transportation sector and transforming public transit. Some transit observers envision future public transit to be integrated transit systems with fixed-route services running along major corridors and on-demand ridesharing services covering lower-density areas. A switch from a conventional fixed-route service model to this kind of integrated mobility-on-demand transit system, however, may elicit varied responses from local residents. This paper evaluates traveler preferences for a proposed integrated mobility-on-demand transit system versus the existing fixed-route system, with a particular focus on disadvantaged travelers. We conducted a survey in two low-resource communities in the United States, namely, Detroit and Ypsilanti, Michigan. A majority of survey respondents preferred a mobility-on-demand transit system over a fixed-route one. Based on ordered logit model outputs, we found a stronger preference for mobility-on-demand transit among males, college graduates, individuals who have never heard of or used ride-hailing before, and individuals who currently receive inferior transit services. By contrast, preferences varied little by age, income, race, or disability status. The most important benefit of a mobility-on-demand transit system perceived by the survey respondents is enhanced transit accessibility to different destinations, whereas their major concerns include the need to actively request rides, possible transit-fare increases, and potential technological failures. Addressing the concerns of female riders and accommodating the needs of less technology-proficient individuals should be major priorities for transit agencies that are considering mobility-on-demand initiatives., Comment: 27 pages, 2 figures