Search

With the rise of prolific ChatGPT, the risk and consequences of AI-generated text has increased alarmingly. To address the inevitable question of ownership attribution for AI-generated artifacts, the US Copyright Office released a statement stating that 'If a work's traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it'. Furthermore, both the US and the EU governments have recently drafted their initial proposals regarding the regulatory framework for AI. Given this cynosural spotlight on generative AI, AI-generated text detection (AGTD) has emerged as a topic that has already received immediate attention in research, with some initial methods having been proposed, soon followed by emergence of techniques to bypass detection. This paper introduces the Counter Turing Test (CT^2), a benchmark consisting of techniques aiming to offer a comprehensive evaluation of the robustness of existing AGTD techniques. Our empirical findings unequivocally highlight the fragility of the proposed AGTD methods under scrutiny. Amidst the extensive deliberations on policy-making for regulating AI development, it is of utmost importance to assess the detectability of content generated by LLMs. Thus, to establish a quantifiable spectrum facilitating the evaluation and ranking of LLMs according to their detectability levels, we propose the AI Detectability Index (ADI). We conduct a thorough examination of 15 contemporary LLMs, empirically demonstrating that larger LLMs tend to have a higher ADI, indicating they are less detectable compared to smaller LLMs. We firmly believe that ADI holds significant value as a tool for the wider NLP community, with the potential to serve as a rubric in AI-related policy-making., Comment: EMNLP 2023 Main

Analyzing memes on the internet has emerged as a crucial endeavor due to the impact this multi-modal form of content wields in shaping online discourse. Memes have become a powerful tool for expressing emotions and sentiments, possibly even spreading hate and misinformation, through humor and sarcasm. In this paper, we present the overview of the Memotion 3 shared task, as part of the DeFactify 2 workshop at AAAI-23. The task released an annotated dataset of Hindi-English code-mixed memes based on their Sentiment (Task A), Emotion (Task B), and Emotion intensity (Task C). Each of these is defined as an individual task and the participants are ranked separately for each task. Over 50 teams registered for the shared task and 5 made final submissions to the test set of the Memotion 3 dataset. CLIP, BERT modifications, ViT etc. were the most popular models among the participants along with approaches such as Student-Teacher model, Fusion, and Ensembling. The best final F1 score for Task A is 34.41, Task B is 79.77 and Task C is 59.82., Comment: Defactify2 @AAAI 2023

Seismology has witnessed significant advancements in recent years with the application of deep learning methods to address a broad range of problems. These techniques have demonstrated their remarkable ability to effectively extract statistical properties from extensive datasets, surpassing the capabilities of traditional approaches to an extent. In this study, we present SAIPy, an open source Python package specifically developed for fast data processing by implementing deep learning. SAIPy offers solutions for multiple seismological tasks, including earthquake detection, magnitude estimation, seismic phase picking, and polarity identification. We introduce upgraded versions of previously published models such as CREIMERT capable of identifying earthquakes with an accuracy above 99.8 percent and a root mean squared error of 0.38 unit in magnitude estimation. These upgraded models outperform state of the art approaches like the Vision Transformer network. SAIPy provides an API that simplifies the integration of these advanced models, including CREIMERT, DynaPickerv2, and PolarCAP, along with benchmark datasets. The package has the potential to be used for real time earthquake monitoring to enable timely actions to mitigate the impact of seismic events. Ongoing development efforts aim to enhance the performance of SAIPy and incorporate additional features that enhance exploration efforts, and it also would be interesting to approach the retraining of the whole package as a multi-task learning problem.

With social media usage growing exponentially in the past few years, fake news has also become extremely prevalent. The detrimental impact of fake news emphasizes the need for research focused on automating the detection of false information and verifying its accuracy. In this work, we present the outcome of the Factify 2 shared task, which provides a multi-modal fact verification and satire news dataset, as part of the DeFactify 2 workshop at AAAI'23. The data calls for a comparison based approach to the task by pairing social media claims with supporting documents, with both text and image, divided into 5 classes based on multi-modal relations. In the second iteration of this task we had over 60 participants and 9 final test-set submissions. The best performances came from the use of DeBERTa for text and Swinv2 and CLIP for image. The highest F1 score averaged for all five classes was 81.82%., Comment: Defactify2 @AAAI 2023

Combating disinformation is one of the burning societal crises -- about 67% of the American population believes that disinformation produces a lot of uncertainty, and 10% of them knowingly propagate disinformation. Evidence shows that disinformation can manipulate democratic processes and public opinion, causing disruption in the share market, panic and anxiety in society, and even death during crises. Therefore, disinformation should be identified promptly and, if possible, mitigated. With approximately 3.2 billion images and 720,000 hours of video shared online daily on social media platforms, scalable detection of multimodal disinformation requires efficient fact verification. Despite progress in automatic text-based fact verification (e.g., FEVER, LIAR), the research community lacks substantial effort in multimodal fact verification. To address this gap, we introduce FACTIFY 3M, a dataset of 3 million samples that pushes the boundaries of the domain of fact verification via a multimodal fake news dataset, in addition to offering explainability through the concept of 5W question-answering. Salient features of the dataset include: (i) textual claims, (ii) ChatGPT-generated paraphrased claims, (iii) associated images, (iv) stable diffusion-generated additional images (i.e., visual paraphrases), (v) pixel-level image heatmap to foster image-text explainability of the claim, (vi) 5W QA pairs, and (vii) adversarial fake news stories., Comment: arXiv admin note: text overlap with arXiv:2305.04329

Word embeddings, i.e., semantically meaningful vector representation of words, are largely influenced by the distributional hypothesis "You shall know a word by the company it keeps" (Harris, 1954), whereas modern prediction-based neural network embeddings rely on design choices and hyperparameter optimization. Word embeddings like Word2Vec, GloVe etc. well capture the contextuality and real-world analogies but contemporary convolution-based image embeddings such as VGGNet, AlexNet, etc. do not capture contextual knowledge. The popular king-queen analogy does not hold true for most commonly used vision embeddings. In this paper, we introduce a pre-trained joint embedding (JE), named IMAGINATOR, trained on 21K distinct image objects level from 1M image+text pairs. JE is a way to encode multimodal data into a vector space where the text modality serves as the ground-ing key, which the complementary modality (in this case, the image) is anchored with. IMAGINATOR encapsulates three individual representations: (i) object-object co-location, (ii) word-object co-location, and (iii) word-object correlation. These three ways capture complementary aspects of the two modalities which are further combined to obtain the final JEs. Generated JEs are intrinsically evaluated to assess how well they capture the contextuality and real-world analogies. We also evaluate pre-trained IMAGINATOR JEs on three downstream tasks: (i) image captioning, (ii) Image2Tweet, and (iii) text-based image retrieval. IMAGINATOR establishes a new standard on the aforementioned down-stream tasks by outperforming the current SoTA on all the selected tasks. IMAGINATOR will be made publicly available. The codes are available at https://github.com/varunakk/IMAGINATOR

Automatic fact verification has received significant attention recently. Contemporary automatic fact-checking systems focus on estimating truthfulness using numerical scores which are not human-interpretable. A human fact-checker generally follows several logical steps to verify a verisimilitude claim and conclude whether its truthful or a mere masquerade. Popular fact-checking websites follow a common structure for fact categorization such as half true, half false, false, pants on fire, etc. Therefore, it is necessary to have an aspect-based (delineating which part(s) are true and which are false) explainable system that can assist human fact-checkers in asking relevant questions related to a fact, which can then be validated separately to reach a final verdict. In this paper, we propose a 5W framework (who, what, when, where, and why) for question-answer-based fact explainability. To that end, we present a semi-automatically generated dataset called FACTIFY-5WQA, which consists of 391, 041 facts along with relevant 5W QAs - underscoring our major contribution to this paper. A semantic role labeling system has been utilized to locate 5Ws, which generates QA pairs for claims using a masked language model. Finally, we report a baseline QA system to automatically locate those answers from evidence documents, which can serve as a baseline for future research in the field. Lastly, we propose a robust fact verification system that takes paraphrased claims and automatically validates them. The dataset and the baseline model are available at https: //github.com/ankuranii/acl-5W-QA, Comment: Accepted at ACL main conference 2023

The internet gives the world an open platform to express their views and share their stories. While this is very valuable, it makes fake news one of our society's most pressing problems. Manual fact checking process is time consuming, which makes it challenging to disprove misleading assertions before they cause significant harm. This is he driving interest in automatic fact or claim verification. Some of the existing datasets aim to support development of automating fact-checking techniques, however, most of them are text based. Multi-modal fact verification has received relatively scant attention. In this paper, we provide a multi-modal fact-checking dataset called FACTIFY 2, improving Factify 1 by using new data sources and adding satire articles. Factify 2 has 50,000 new data instances. Similar to FACTIFY 1.0, we have three broad categories - support, no-evidence, and refute, with sub-categories based on the entailment of visual and textual data. We also provide a BERT and Vison Transformer based baseline, which achieves 65% F1 score in the test set. The baseline codes and the dataset will be made available at https://github.com/surya1701/Factify-2.0., Comment: Defactify2 @AAAI2023

Memes are the new-age conveyance mechanism for humor on social media sites. Memes often include an image and some text. Memes can be used to promote disinformation or hatred, thus it is crucial to investigate in details. We introduce Memotion 3, a new dataset with 10,000 annotated memes. Unlike other prevalent datasets in the domain, including prior iterations of Memotion, Memotion 3 introduces Hindi-English Codemixed memes while prior works in the area were limited to only the English memes. We describe the Memotion task, the data collection and the dataset creation methodologies. We also provide a baseline for the task. The baseline code and dataset will be made available at https://github.com/Shreyashm16/Memotion-3.0, Comment: Defactify2 @AAAI

Seismic phase picking and magnitude estimation are essential components of real time earthquake monitoring and earthquake early warning systems. Reliable phase picking enables the timely detection of seismic wave arrivals, facilitating rapid earthquake characterization and early warning alerts. Accurate magnitude estimation provides crucial information about the size of an earthquake and potential impact. Together, these steps contribute to effective earthquake monitoring, enhancing our ability to implement appropriate response measures in seismically active regions and mitigate risks. In this study, we explore the potential of deep learning in real time earthquake monitoring. To that aim, we begin by introducing DynaPicker which leverages dynamic convolutional neural networks to detect seismic body wave phases. Subsequently, DynaPicker is employed for seismic phase picking on continuous seismic recordings. To showcase the efficacy of Dynapicker, several open source seismic datasets including window format data and continuous seismic data are used for seismic phase identification, and arrival time picking. Additionally,the robustness of DynaPicker in classifying seismic phases was tested on the low magnitude seismic data polluted by noise. Finally, the phase arrival time information is integrated into a previously published deep learning model for magnitude estimation. This workflow is then applied and tested on the continuous recording of the aftershock sequences following the Turkey earthquake to detect the earthquakes, seismic phase picking and estimate the magnitude of the corresponding event. The results obtained in this case study exhibit a high level of reliability in detecting the earthquakes and estimating the magnitude of aftershocks following the Turkey earthquake.

The detection and rapid characterisation of earthquake parameters such as magnitude are of prime importance in seismology, particularly in applications such as Earthquake Early Warning (EEW). Traditionally, algorithms such as STA/LTA are used for event detection, while frequency or amplitude domain parameters calculated from 1-3 seconds of first P-arrival data are sometimes used to provide a first estimate of (body wave) magnitude. Owing to extensive involvement of human experts in parameter determination, these approaches are often found to be insufficient. Moreover, these methods are sensitive to the signal to noise ratio and may often lead to false or missed alarms depending on the choice of parameters. We, therefore, propose a multitasking deep learning model the Convolutional Recurrent model for Earthquake Identification and Magnitude Estimation (CREIME) that: (i) detects the first earthquake signal, from background seismic noise, (ii) determines first P arrival time as well as (iii) estimates the magnitude using the raw 3-component waveform data from a single station as model input. Considering, speed is of essence in EEW, we use up to two seconds of P-wave information which, to the best of our knowledge, is a significantly smaller data window (5 second window with up to of P wave data) compared to the previous studies. To examine the robustness of CREIME we test it on two independent datasets and find that it achieves an average accuracy of 98 percent for event vs noise discrimination and is able to estimate first P arrival time and local magnitude with average root mean squared errors of 0.13 seconds and 0.65 units, respectively. We also compare CREIME architecture with architectures of other baseline models, by training them on the same data, and also with traditional algorithms such as STA/LTA, and show that our architecture outperforms these methods.

The rapid characterisation of earthquake parameters such as its magnitude is at the heart of Earthquake Early Warning (EEW). In traditional EEW methods the robustness in the estimation of earthquake parameters have been observed to increase with the length of input data. Since time is a crucial factor in EEW applications, in this paper we propose a deep learning based magnitude classifier and, further we investigate the effect of using five different durations of seismic waveform data after first P wave arrival of length 1s, 3s, 10s, 20s and 30s. This is accomplished by testing the performance of the proposed model that combines Convolution and Bidirectional Long-Short Term Memory units to classify waveforms based on their magnitude into three classes "noise", "low magnitude events" and "high magnitude events". Herein, any earthquake signal with magnitude equal to or above 5.0 is labelled as high magnitude. We show that the variation in the results produced by changing the length of the data, is no more than the inherent randomness in the trained models, due to their initialisation.

Many active volcanoes in the world exhibit Strombolian activity, which is typically characterized by relatively frequent mild events and also by rare and much more destructive major explosions and paroxysms. Detailed analyses of past major and minor events can help to understand the eruptive behavior of the volcano and the underlying physical and chemical processes. Catalogs of volcanic eruptions may be established using continuous seismic recordings at stations in the proximity of volcanoes. However, in many cases, the analysis of the recordings relies heavily on the manual picking of events by human experts. Recently developed Machine Learning-based approaches require large training data sets which may not be available a priori. Here, we propose an alternative automated approach: the Adaptive-Window Volcanic Event Selection Analysis Module (AWESAM). This process of creating event catalogs consists of three main steps: (i) identification of potential volcanic events based on squared ground-velocity amplitudes, an adaptive MaxFilter, and a prominence threshold. (ii) catalog consolidation by comparing and verification of the initial detections based on recordings from two different seismic stations. (iii) identification and exclusion of signals from regional tectonic earthquakes. The software package is applied to publicly accessible continuous seismic recordings from two almost equidistant stations at Stromboli volcano in Italy. We tested AWESAM by comparison with a hand-picked catalog and found that around 95 percent of the eruptions with a signal-to-noise ratio above three are detected. In a first application, we derive a new amplitude-frequency relationship from over 290.000 volcanic events at Stromboli during 2019-2020. The module allows for a straightforward generalization and application to other volcanoes worldwide.

Earthquake detection and seismic phase picking not only play a crucial role in travel time estimation of body waves(P and S waves) but also in the localisation of the epicenter of the corresponding event. Generally, manual phase picking is a trustworthy and the optimum method to determine the phase arrival time, however, its capacity is restricted by available resources and time. Moreover, noisy seismic data renders an additional critical challenge for fast and accurate phase picking. In this study, a deep learning based model, EPick, is proposed which benefits both from U shaped neural network (also called UNet)and attention mechanism, as a strong alternative for seismic event detection and phase picking. On one hand, the utilization of UNet structure enables addressing different levels of deep features. On the other hand, attention mechanism promotes the decoder in the UNet structure to focus on the efficient exploitation of the low-resolution features learned from the encoder part to achieve precise phase picking. Extensive experimental results demonstrate that EPick achieves better performance over the benchmark method, and show the models robustness when tested on a different seismic dataset.

Spatial and temporal data for earthquakes with magnitude M greater than or equal to 6.5 can provide crucial information about the seismic history and potential for large earthquakes in a region. We analyzed approximately 313,500 events that occurred in the Sunda-arc region during the last 56 years, from 1964 to 2020, reported by the International Seismological Center. We report a persistent increase in the annual number of the events with mb greater than or equal to 6.5. We tested this increase against the null hypothesis and discarded the possibility of the increase being due to random groupings. The trend given by Auto-Regressive Integrated Moving Average suggests continuing increase of such large-magnitude events in the region during the next decade. At the same time, the computed Gutenberg Richter b value shows anomalies that can be related to the occurrence of the mega 2004 Sumatra earthquake, and to possible state of high tectonic stress in the eastern parts of the region.