Search

Correctional education is a prevalent form of rehabilitation programming for prisoners in the United States. There is limited evidence, however, about the labor market returns to credentials received while incarcerated. Using incarceration, educational, and labor market data in Washington State, we study the labor market returns to GEDs and short-term vocational certificates earned in prison. We identify the returns to credentials by a difference-indifferences design that compares changes in earnings and employment for incarcerated persons who earn a credential to those who enroll in a program but fail to complete a GED or certificate. We estimate that GEDs increase post-incarceration earnings by about $450 per quarter and that vocational certificates increase earnings by about $250 per quarter. Degree completers have higher hourly wages, are more likely to be employed, and work more hours following release. For vocational programs, earnings increases are driven by certificates in construction and manufacturing. [The research presented presented in this report uses confidential data from the Education Research and Data Center (ERDC) located within the Washington Office of Financial Management (OFM).]

Graduates of special education teacher education programs can teach in a range of special education settings, raising the potential that their training can occur in very different settings than where they find their first jobs. We follow 263 completers of Moderate Disabilities programs in Massachusetts from their field placements to their early-career teaching positions and study the characteristics of their field placements and the degree to which these are aligned with their early-career teaching positions. We also assess the degree to which alignment is associated with early-career teacher turnover. We found that many of these teachers student-taught in an inclusive setting but were hired into a self-contained special education setting and vice versa, and teachers who experienced this misalignment were more likely to leave the workforce early in their careers. Teachers who student taught with a supervising practitioner without a special education license were also more likely to leave early. Findings suggest that teachers training to educate students with learning disabilities should student teach in a setting that is aligned with where they are likely to be hired, and with a supervising practitioner who is trained in special education.

This publication was prepared as part of the Cedefop project The shift to learning outcomes: rhetoric or reality. The purpose of this research is to analyse the conceptual, structural and political factors influencing the transformation of intended learning outcomes into achieved learning outcomes. It is considered as a first step in a long-term research strategy aiming to understand better the conditions for high quality vocational education, training and learning. The research focuses on initial vocational education and training, in schools and apprenticeships, in the 27 Member States of the EU as well as Iceland and Norway. The research is divided into five separate but interlinked themes: (1) Addressing the influence of learning outcomes on pedagogical theory and tools; (2) Focusing on the influence of learning outcomes-based curricula on teaching practices (in school-based programmes); (3) Examining the influence of learning outcomes-based curricula in company training (part of apprenticeship programmes that takes place in companies); (4) Mapping and analysing the influence of learning outcomes on assessment; and (5) Developing suggestions for the way forward supporting stakeholders and policy-makers in addressing future challenges and opportunities in this area. This report examines how the learning outcomes approach is embedded in and promoted by theories of teaching and learning (epistemology, didactics, pedagogy). It aims to analyse selected theories and the way these are presented to teacher training institutions in selected regions/countries. This allows for a better understanding of the explicit and implicit assumptions made regarding the role and relevance of learning outcomes. Results of the research illustrate differences across countries in whether and how the learning outcomes approach is embedded in theories underpinning VET teacher training programmes.

Average-wage-by-major statistics have become widely available to students interested in the economic ramifications of their college major choice. However, earning a major with higher average wages does not necessarily lead individual students to higher-paying careers. This essay combines literature review with novel analysis of longitudinal student outcomes to discuss how students use average-wage-by-major statistics and document seven reasons that they may differ, sharply in some cases, from the causal wage effects of major choice. I focus on the ramifications of two-sided non-random selection into college majors, mismeasurement of longitudinal student outcomes, and failures of extrapolation between available statistics and student interests. While large differences in average wages by major are likely to indicate causal ordinal differences between fields, small differences are probably best ignored even by students with strong interest in the economic consequences of their major choices. This essay is adapted from Chapter 6 of "Metrics that Matter: Counting What's Really Important to College Students."

This paper describes a system designed to distinguish between AI-generated and human-written scientific excerpts in the DAGPap24 competition hosted within the Fourth Workshop on Scientific Document Processing. In this competition the task is to find artificially generated token-level text fragments in documents of a scientific domain. Our work focuses on the use of a multi-task learning architecture with two heads. The application of this approach is justified by the specificity of the task, where class spans are continuous over several hundred characters. We considered different encoder variations to obtain a state vector for each token in the sequence, as well as a variation in splitting fragments into tokens to further feed into the input of a transform-based encoder. This approach allows us to achieve a 9% quality improvement relative to the baseline solution score on the development set (from 0.86 to 0.95) using the average macro F1-score, as well as a score of 0.96 on a closed test part of the dataset from the competition.

Scientific literature is typically dense, requiring significant background knowledge and deep comprehension for effective engagement. We introduce SciDQA, a new dataset for reading comprehension that challenges LLMs for a deep understanding of scientific articles, consisting of 2,937 QA pairs. Unlike other scientific QA datasets, SciDQA sources questions from peer reviews by domain experts and answers by paper authors, ensuring a thorough examination of the literature. We enhance the dataset's quality through a process that carefully filters out lower quality questions, decontextualizes the content, tracks the source document across different versions, and incorporates a bibliography for multi-document question-answering. Questions in SciDQA necessitate reasoning across figures, tables, equations, appendices, and supplementary materials, and require multi-document reasoning. We evaluate several open-source and proprietary LLMs across various configurations to explore their capabilities in generating relevant and factual responses. Our comprehensive evaluation, based on metrics for surface-level similarity and LLM judgements, highlights notable performance discrepancies. SciDQA represents a rigorously curated, naturally derived scientific QA dataset, designed to facilitate research on complex scientific text understanding., Comment: 18 pages, Accepted to EMNLP 2024

The European Union's Artificial Intelligence Act (AI Act) introduces comprehensive guidelines for the development and oversight of Artificial Intelligence (AI) and Machine Learning (ML) systems, with significant implications for Graph Neural Networks (GNNs). This paper addresses the unique challenges posed by the AI Act for GNNs, which operate on complex graph-structured data. The legislation's requirements for data management, data governance, robustness, human oversight, and privacy necessitate tailored strategies for GNNs. Our study explores the impact of these requirements on GNN training and proposes methods to ensure compliance. We provide an in-depth analysis of bias, robustness, explainability, and privacy in the context of GNNs, highlighting the need for fair sampling strategies and effective interpretability techniques. Our contributions fill the research gap by offering specific guidance for GNNs under the new legislative framework and identifying open questions and future research directions.

We generalize Rock Paper Scissors to complete directed graphs, or tournaments, on $n$ vertices. Properties of the mixed-strategy Nash equilibria of these tournaments are discussed, particularly those with Nash equilibria where all of the strategies have a nonzero probability. We find graph-theoretic properties of such games and tabulate them for $n \leq 7$., Comment: 18 pages

The large-scale training of multi-modal models on data scraped from the web has shown outstanding utility in infusing these models with the required world knowledge to perform effectively on multiple downstream tasks. However, one downside of scraping data from the web can be the potential sacrifice of the benchmarks on which the abilities of these models are often evaluated. To safeguard against test data contamination and to truly test the abilities of these foundation models we propose LiveXiv: A scalable evolving live benchmark based on scientific ArXiv papers. LiveXiv accesses domain-specific manuscripts at any given timestamp and proposes to automatically generate visual question-answer pairs (VQA). This is done without any human-in-the-loop, using the multi-modal content in the manuscripts, like graphs, charts, and tables. Moreover, we introduce an efficient evaluation approach that estimates the performance of all models on the evolving benchmark using evaluations of only a subset of models. This significantly reduces the overall evaluation cost. We benchmark multiple open and proprietary Large Multi-modal Models (LMMs) on the first version of our benchmark, showing its challenging nature and exposing the models true abilities, avoiding contamination. Lastly, in our commitment to high quality, we have collected and evaluated a manually verified subset. By comparing its overall results to our automatic annotations, we have found that the performance variance is indeed minimal (<2.5%). Our dataset is available online on HuggingFace, and our code will be available here.

This paper provides an overview of the techniques employed by Hibikino-Musashi@Home, which intends to participate in the domestic standard platform league. The team has developed a dataset generator for training a robot vision system and an open-source development environment running on a Human Support Robot simulator. The large language model powered task planner selects appropriate primitive skills to perform the task requested by users. The team aims to design a home service robot that can assist humans in their homes and continuously attends competitions to evaluate and improve the developed system.

Peer review is a critical process for ensuring the integrity of published scientific research. Confidence in this process is predicated on the assumption that experts in the relevant domain give careful consideration to the merits of manuscripts which are submitted for publication. With the recent rapid advancements in the linguistic capabilities of large language models (LLMs), a new potential risk to the peer review process is that negligent reviewers will rely on LLMs to perform the often time consuming process of reviewing a paper. In this study, we investigate the ability of existing AI text detection algorithms to distinguish between peer reviews written by humans and different state-of-the-art LLMs. Our analysis shows that existing approaches fail to identify many GPT-4o written reviews without also producing a high number of false positive classifications. To address this deficiency, we propose a new detection approach which surpasses existing methods in the identification of GPT-4o written peer reviews at low levels of false positive classifications. Our work reveals the difficulty of accurately identifying AI-generated text at the individual review level, highlighting the urgent need for new tools and methods to detect this type of unethical application of generative AI.

This study investigates efficient deduplication techniques for a large NLP dataset of economic research paper titles. We explore various pairing methods alongside established distance measures (Levenshtein distance, cosine similarity) and a sBERT model for semantic evaluation. Our findings suggest a potentially low prevalence of duplicates based on the observed semantic similarity across different methods. Further exploration with a human-annotated ground truth set is completed for a more conclusive assessment. The result supports findings from the NLP, LLM based distance metrics., Comment: 6 pages, 1 figure

This paper investigates the performance of several representative large models in the field of literature recommendation and explores potential biases. The results indicate that while some large models' recommendations can be somewhat satisfactory after simple manual screening, overall, the accuracy of these models in specific literature recommendation tasks is generally moderate. Additionally, the models tend to recommend literature that is timely, collaborative, and expands or deepens the field. In scholar recommendation tasks. There is no evidence to suggest that LLMs exacerbate inequalities related to gender, race, or the level of development of countries., Comment: There are some aspects in the original research that need to be supplemented with robustness checks

There is mounting evidence that species interactions often involve long-term memory, with highly-varying waiting times between successive events and long-range temporal correlations. Accounting for memory undermines the common Markovian assumption, and dramatically impacts key ingredients of population dynamics including birth, foraging, predation, and competition processes. Here, we investigate a critical aspect of population dynamics, namely non-Markovian multi-species competition. This is done in the realm of the zero-sum rock-paper-scissors (zRPS) model that is broadly used in the life sciences to metaphorically describe cyclic competition between three interacting species. We develop a general non-Markovian formalism for multi-species dynamics, allowing us to determine the regions of the parameter space where each species dominates. In particular, when the dynamics are Markovian, the waiting times are exponentially distributed and the fate of the zRPS model in large well-mixed populations follows the so-called ``law of the weakest'' (LOW), predicting that the species with the lowest growth rate is the most likely to prevail. We demonstrate that the survival behavior and LOW of the zRPS model are critically affected by non-exponential waiting time distributions, and especially, by their coefficient of variation. Our findings provide key insight into the influence of long waiting times on non-Markovian evolutionary processes., Comment: 10 pages, 5 figures + appendices

As the body of academic literature continues to grow, researchers face increasing difficulties in effectively searching for relevant resources. Existing databases and search engines often fall short of providing a comprehensive and contextually relevant collection of academic literature. To address this issue, we propose a novel framework that leverages Natural Language Processing (NLP) techniques. This framework automates the retrieval, summarization, and clustering of academic literature within a specific research domain. To demonstrate the effectiveness of our approach, we introduce CyLit, an NLP-powered repository specifically designed for the cyber risk literature. CyLit empowers researchers by providing access to context-specific resources and enabling the tracking of trends in the dynamic and rapidly evolving field of cyber risk. Through the automatic processing of large volumes of data, our NLP-powered solution significantly enhances the efficiency and specificity of academic literature searches. We compare the literature categorization results of CyLit to those presented in survey papers or generated by ChatGPT, highlighting the distinctive insights this tool provides into cyber risk research literature. Using NLP techniques, we aim to revolutionize the way researchers discover, analyze, and utilize academic resources, ultimately fostering advancements in various domains of knowledge.