Search

Recent advancements in text-to-image diffusion models have brought them to the public spotlight, becoming widely accessible and embraced by everyday users. However, these models have been shown to generate harmful content such as not-safe-for-work (NSFW) images. While approaches have been proposed to erase such abstract concepts from the models, jail-breaking techniques have succeeded in bypassing such safety measures. In this paper, we propose TraSCE, an approach to guide the diffusion trajectory away from generating harmful content. Our approach is based on negative prompting, but as we show in this paper, conventional negative prompting is not a complete solution and can easily be bypassed in some corner cases. To address this issue, we first propose a modification of conventional negative prompting. Furthermore, we introduce a localized loss-based guidance that enhances the modified negative prompting technique by steering the diffusion trajectory. We demonstrate that our proposed method achieves state-of-the-art results on various benchmarks in removing harmful content including ones proposed by red teams; and erasing artistic styles and objects. Our proposed approach does not require any training, weight modifications, or training data (both image or prompt), making it easier for model owners to erase new concepts.

Diffusion models are prone to exactly reproduce images from the training data. This exact reproduction of the training data is concerning as it can lead to copyright infringement and/or leakage of privacy-sensitive information. In this paper, we present a novel way to understand the memorization phenomenon, and propose a simple yet effective approach to mitigate it. We argue that memorization occurs because of an attraction basin in the denoising process which steers the diffusion trajectory towards a memorized image. However, this can be mitigated by guiding the diffusion trajectory away from the attraction basin by not applying classifier-free guidance until an ideal transition point occurs from which classifier-free guidance is applied. This leads to the generation of non-memorized images that are high in image quality and well-aligned with the conditioning mechanism. To further improve on this, we present a new guidance technique, \emph{opposite guidance}, that escapes the attraction basin sooner in the denoising process. We demonstrate the existence of attraction basins in various scenarios in which memorization occurs, and we show that our proposed approach successfully mitigates memorization.

Postdoctoral training is a career stage often described as a demanding and anxiety-laden time when many promising PhDs see their academic dreams slip away due to circumstances beyond their control. We use a unique data set of academic publishing and careers to chart the more or less successful postdoctoral paths. We build a measure of academic success on the citation patterns two to five years into a faculty career. Then, we monitor how students' postdoc positions -- in terms of relocation, change of topic, and early well-cited papers -- relate to their early-career success. One key finding is that the postdoc period seems more important than the doctoral training to achieve this form of success. This is especially interesting in light of the many studies of academic faculty hiring that link Ph.D. granting institutions and hires, omitting the postdoc stage. Another group of findings can be summarized as a Goldilocks principle: it seems beneficial to change one's direction, but not too much.

This article explores the theoretical underpinnings of -- and practical results for -- topological and geometric features of data collected by synthetic aperture sonar systems. We prove a strong theoretical guarantee about the structure of the space of echos (the signature space) that is relevant for classification, and that this structure is measurable in laboratory conditions. The guarantee makes minimal assumptions about the sonar platform's trajectory, and establishes that the signature space divides neatly into topological features based upon the number of prominent echos and their placement, and geometric features that capture their corresponding sonar cross section., Comment: 34 pages, 16 figures

Social robot navigation (SRN) is a relevant problem that involves navigating a pedestrian-rich environment in a socially acceptable manner. It is an essential part of making social robots effective in pedestrian-rich settings. The use cases of such robots could vary from companion robots to warehouse robots to autonomous wheelchairs. In recent years, deep reinforcement learning has been increasingly used in research on social robot navigation. Our work introduces CAMRL (Context-Aware Mamba-based Reinforcement Learning). Mamba is a new deep learning-based State Space Model (SSM) that has achieved results comparable to transformers in sequencing tasks. CAMRL uses Mamba to determine the robot's next action, which maximizes the value of the next state predicted by the neural network, enabling the robot to navigate effectively based on the rewards assigned. We evaluate CAMRL alongside existing solutions (CADRL, LSTM-RL, SARL) using a rigorous testing dataset which involves a variety of densities and environment behaviors based on ORCA and SFM, thus, demonstrating that CAMRL achieves higher success rates, minimizes collisions, and maintains safer distances from pedestrians. This work introduces a new SRN planner, showcasing the potential for deep-state space models for robot navigation.

Open Domain Question Answering (ODQA) within natural language processing involves building systems that answer factual questions using large-scale knowledge corpora. Recent advances stem from the confluence of several factors, such as large-scale training datasets, deep learning techniques, and the rise of large language models. High-quality datasets are used to train models on realistic scenarios and enable the evaluation of the system on potentially unseen data. Standardized metrics facilitate comparisons between different ODQA systems, allowing researchers to objectively track advancements in the field. Our study presents a thorough examination of the current landscape of ODQA benchmarking by reviewing 52 datasets and 20 evaluation techniques across textual and multimodal modalities. We introduce a novel taxonomy for ODQA datasets that incorporates both the modality and difficulty of the question types. Additionally, we present a structured organization of ODQA evaluation metrics along with a critical analysis of their inherent trade-offs. Our study aims to empower researchers by providing a framework for the robust evaluation of modern question-answering systems. We conclude by identifying the current challenges and outlining promising avenues for future research and development., Comment: 22 pages, 13 tables, 7 figures

We demonstrate absorber-free passive and hybrid mode-locking at sub-GHz repetition rates using a hybrid integrated extended cavity diode laser around 1550 nm. The laser is based on InP as gain medium and a long Si$_3$N$_4$ feedback circuit, with three highly frequency selective microring resonators. The feedback resonators not only increases the cavity length up to 0.6 m to achieve sub-GHz repetition rates but also serve as a dispersive narrowband mirror for sharp spectral filtering, which enables Fourier domain mode-locking. We observe passive mode-locking with repetition rates below 500 MHz, with $\approx$ 15 comb lines at around 0.2 mW total power. To stabilize the repetition rate, hybrid mode-locking is demonstrated by weak RF modulation of the diode current at frequencies around 500 MHz. The RF injection reduces the Lorentzian linewidth component from 8.9 kHz to a detection limited value around 300 mHz. To measure the locking range of the repetition rate, the injected RF frequency is tuned with regard to the passive mode-locking frequency and the injected RF power is varied. The locking range increases approximately as a square-root function of the injected RF power. At 1 mW injection a wide locking range of about 80 MHz is obtained. We observe the laser maintaining stable mode-locking also when the DC diode pump current is increased from 40 mA to 190 mA, provided that the cavity length is maintained constant with thermo-refractive tuning., Comment: This manuscript is under review for publication with optics express

Through a large-scale study over diverse face images, we show that facial attribute editing using modern generative AI models can severely degrade automated face recognition systems. This degradation persists even with identity-preserving generative models. To mitigate this issue, we propose two novel techniques for local and global attribute editing. We empirically ablate twenty-six facial semantic, demographic and expression-based attributes that have been edited using state-of-the-art generative models, and evaluate them using ArcFace and AdaFace matchers on CelebA, CelebAMaskHQ and LFW datasets. Finally, we use LLaVA, an emerging visual question-answering framework for attribute prediction to validate our editing techniques. Our methods outperform the current state-of-the-art at facial editing (BLIP, InstantID) while improving identity retention by a significant extent., Comment: Under review

The rise of AI voice-cloning technology, particularly audio Real-time Deepfakes (RTDFs), has intensified social engineering attacks by enabling real-time voice impersonation that bypasses conventional enrollment-based authentication. To address this, we propose PITCH, a robust challenge-response method to detect and tag interactive deepfake audio calls. We developed a comprehensive taxonomy of audio challenges based on the human auditory system, linguistics, and environmental factors, yielding 20 prospective challenges. These were tested against leading voice-cloning systems using a novel dataset comprising 18,600 original and 1.6 million deepfake samples from 100 users. PITCH's prospective challenges enhanced machine detection capabilities to 88.7% AUROC score on the full unbalanced dataset, enabling us to shortlist 10 functional challenges that balance security and usability. For human evaluation and subsequent analyses, we filtered a challenging, balanced subset. On this subset, human evaluators independently scored 72.6% accuracy, while machines achieved 87.7%. Acknowledging that call environments require higher human control, we aided call receivers in making decisions with them using machines. Our solution uses an early warning system to tag suspicious incoming calls as "Deepfake-likely." Contrary to prior findings, we discovered that integrating human intuition with machine precision offers complementary advantages. Our solution gave users maximum control and boosted detection accuracy to 84.5%. Evidenced by this jump in accuracy, PITCH demonstrated the potential for AI-assisted pre-screening in call verification processes, offering an adaptable and usable approach to combat real-time voice-cloning attacks. Code to reproduce and access data at \url{https://github.com/mittalgovind/PITCH-Deepfakes}.

In this commentary, we discuss the evolving nature of search engines, as they begin to generate, index, and distribute content created by generative artificial intelligence (GenAI). Our discussion highlights challenges in the early stages of GenAI integration, particularly around factual inconsistencies and biases. We discuss how output from GenAI carries an unwarranted sense of credibility, while decreasing transparency and sourcing ability. Furthermore, search engines are already answering queries with error-laden, generated content, further blurring the provenance of information and impacting the integrity of the information ecosystem. We argue how all these factors could reduce the reliability of search engines. Finally, we summarize some of the active research directions and open questions., Comment: 9 pages, 5 figures

A dictionary attack in a biometric system entails the use of a small number of strategically generated images or templates to successfully match with a large number of identities, thereby compromising security. We focus on dictionary attacks at the template level, specifically the IrisCodes used in iris recognition systems. We present an hitherto unknown vulnerability wherein we mix IrisCodes using simple bitwise operators to generate alpha-mixtures - alpha-wolves (combining a set of "wolf" samples) and alpha-mammals (combining a set of users selected via search optimization) that increase false matches. We evaluate this vulnerability using the IITD, CASIA-IrisV4-Thousand and Synthetic datasets, and observe that an alpha-wolf (from two wolves) can match upto 71 identities @FMR=0.001%, while an alpha-mammal (from two identities) can match upto 133 other identities @FMR=0.01% on the IITD dataset., Comment: 8 pages, 5 figures, 13 tables, Workshop on Manipulation, Adversarial, and Presentation Attacks in Biometrics, Winter Conference on Applications of Computer Vision

The National Cancer Institute (NCI) Image Data Commons (IDC) offers publicly available cancer radiology collections for cloud computing, crucial for developing advanced imaging tools and algorithms. Despite their potential, these collections are minimally annotated; only 4% of DICOM studies in collections considered in the project had existing segmentation annotations. This project increases the quantity of segmentations in various IDC collections. We produced high-quality, AI-generated imaging annotations dataset of tissues, organs, and/or cancers for 11 distinct IDC image collections. These collections contain images from a variety of modalities, including computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET). The collections cover various body parts, such as the chest, breast, kidneys, prostate, and liver. A portion of the AI annotations were reviewed and corrected by a radiologist to assess the performance of the AI models. Both the AI's and the radiologist's annotations were encoded in conformance to the Digital Imaging and Communications in Medicine (DICOM) standard, allowing for seamless integration into the IDC collections as third-party analysis collections. All the models, images and annotations are publicly accessible., Comment: 24 pages; 20 figures

In this critique of the Cognitive Interview (CI), discussion is organized around four themes; (1) the effectiveness of various components of the CI, (2) the relationship between the CI and other interviewing methods such as the Guided Memory Interview, the Standard Interview, and the Structured Interview, (3) different measures of memory performance and (4) the effect of training quality on interviewer performance. We attempt to comment on some of the theoretical and methodological issues to be considered in CI research and the practical considerations relating to the use of the CI in the field.

Information Forensics and Security (IFS) is an active R&D area whose goal is to ensure that people use devices, data, and intellectual properties for authorized purposes and to facilitate the gathering of solid evidence to hold perpetrators accountable. For over a quarter century since the 1990s, the IFS research area has grown tremendously to address the societal needs of the digital information era. The IEEE Signal Processing Society (SPS) has emerged as an important hub and leader in this area, and the article below celebrates some landmark technical contributions. In particular, we highlight the major technological advances on some selected focus areas in the field developed in the last 25 years from the research community and present future trends.

Resection and whole brain radiotherapy (WBRT) are the standards of care for the treatment of patients with brain metastases (BM) but are often associated with cognitive side effects. Stereotactic radiosurgery (SRS) involves a more targeted treatment approach and has been shown to avoid the side effects associated with WBRT. However, SRS requires precise identification and delineation of BM. While many AI algorithms have been developed for this purpose, their clinical adoption has been limited due to poor model performance in the clinical setting. Major reasons for non-generalizable algorithms are the limitations in the datasets used for training the AI network. The purpose of this study was to create a large, heterogenous, annotated BM dataset for training and validation of AI models to improve generalizability. We present a BM dataset of 200 patients with pretreatment T1, T1 post-contrast, T2, and FLAIR MR images. The dataset includes contrast-enhancing and necrotic 3D segmentations on T1 post-contrast and whole tumor (including peritumoral edema) 3D segmentations on FLAIR. Our dataset contains 975 contrast-enhancing lesions, many of which are sub centimeter, along with clinical and imaging feature information. We used a streamlined approach to database-building leveraging a PACS-integrated segmentation workflow., Comment: 12 pages, 2 figures, 1 table

In this study, the first-of-its-kind use of active learning (AL) framework in thermal spray is adapted to improve the prediction accuracy of the in-flight particle characteristics and uses Gaussian Process (GP) ML model as a surrogate that generalises a global solution without necessarily involving physical mechanisms. The AL framework via the Bayesian Optimisation was utilised to: (a) reduce the maximum uncertainty in the given database and (b) reduce local uncertainty around a contrived test point. The initial dataset consists of 26 atmospheric plasma spray (APS) parameters of silicon, aimed at ceramic matrix composites (CMCs) for the next generation of aerospace applications. The maximum uncertainty in the initial dataset was reduced by AL-driven identification of search spaces and conducting six guided spray trails in the identified search spaces. On average, a 52.9% improvement (error reduction) of RMSE and an R2 increase of 8.5% were reported on the predicted in-flight particle velocities and temperatures after the AL-driven optimisation. Furthermore, the Bayesian Optimisation around a contrived test point to predict the best possible characteristics resulted in a three-fold increase in prediction accuracy as compared to the non-optimised prediction. These AL-guided experimental validations not only increase the informativeness of the limited dataset but is adaptable for other thermal spraying methods without necessarily involving physical mechanisms and underlying mechanisms. The use of AL-driven optimisation may drive the thermal spraying towards resource-efficiency and may serve as the first step towards fully digital thermal spraying environments.

Prior to the deep learning era, shape was commonly used to describe the objects. Nowadays, state-of-the-art (SOTA) algorithms in medical imaging are predominantly diverging from computer vision, where voxel grids, meshes, point clouds, and implicit surface models are used. This is seen from numerous shape-related publications in premier vision conferences as well as the growing popularity of ShapeNet (about 51,300 models) and Princeton ModelNet (127,915 models). For the medical domain, we present a large collection of anatomical shapes (e.g., bones, organs, vessels) and 3D models of surgical instrument, called MedShapeNet, created to facilitate the translation of data-driven vision algorithms to medical applications and to adapt SOTA vision algorithms to medical problems. As a unique feature, we directly model the majority of shapes on the imaging data of real patients. As of today, MedShapeNet includes 23 dataset with more than 100,000 shapes that are paired with annotations (ground truth). Our data is freely accessible via a web interface and a Python application programming interface (API) and can be used for discriminative, reconstructive, and variational benchmarks as well as various applications in virtual, augmented, or mixed reality, and 3D printing. Exemplary, we present use cases in the fields of classification of brain tumors, facial and skull reconstructions, multi-class anatomy completion, education, and 3D printing. In future, we will extend the data and improve the interfaces. The project pages are: https://medshapenet.ikim.nrw/ and https://github.com/Jianningli/medshapenet-feedback, Comment: 16 pages

Deep generative models require large amounts of training data. This often poses a problem as the collection of datasets can be expensive and difficult, in particular datasets that are representative of the appropriate underlying distribution (e.g. demographic). This introduces biases in datasets which are further propagated in the models. We present an approach to construct an unbiased generative adversarial network (GAN) from an existing biased GAN by rebalancing the model distribution. We do so by generating balanced data from an existing imbalanced deep generative model using an evolutionary algorithm and then using this data to train a balanced generative model. Additionally, we propose a bias mitigation loss function that minimizes the deviation of the learned class distribution from being equiprobable. We show results for the StyleGAN2 models while training on the Flickr Faces High Quality (FFHQ) dataset for racial fairness and see that the proposed approach improves on the fairness metric by almost 5 times, whilst maintaining image quality. We further validate our approach by applying it to an imbalanced CIFAR10 dataset where we show that we can obtain comparable fairness and image quality as when training on a balanced CIFAR10 dataset which is also twice as large. Lastly, we argue that the traditionally used image quality metrics such as Frechet inception distance (FID) are unsuitable for scenarios where the class distributions are imbalanced and a balanced reference set is not available.

The performance of automated face recognition systems is inevitably impacted by the facial aging process. However, high quality datasets of individuals collected over several years are typically small in scale. In this work, we propose, train, and validate the use of latent text-to-image diffusion models for synthetically aging and de-aging face images. Our models succeed with few-shot training, and have the added benefit of being controllable via intuitive textual prompting. We observe high degrees of visual realism in the generated images while maintaining biometric fidelity measured by commonly used metrics. We evaluate our method on two benchmark datasets (CelebA and AgeDB) and observe significant reduction (~44%) in the False Non-Match Rate compared to existing state-of the-art baselines., Comment: Accepted to appear in International Joint Conference in Biometrics (IJCB) 2023

Retracting academic papers is a fundamental tool of quality control when the validity of papers or the integrity of authors is questioned post-publication. While retractions do not eliminate papers from the record, they have far-reaching consequences for retracted authors and their careers, serving as a visible and permanent signal of potential transgressions. Previous studies have highlighted the adverse effects of retractions on citation counts and coauthors' citations; however, the broader impacts beyond these have not been fully explored. We address this gap leveraging Retraction Watch, the most extensive data set on retractions and link it to Microsoft Academic Graph, a comprehensive data set of scientific publications and their citation networks, and Altmetric that monitors online attention to scientific output. Our investigation focuses on: 1) the likelihood of authors exiting scientific publishing following a retraction, and 2) the evolution of collaboration networks among authors who continue publishing after a retraction. Our empirical analysis reveals that retracted authors, particularly those with less experience, tend to leave scientific publishing in the aftermath of retraction, particularly if their retractions attract widespread attention. We also uncover that retracted authors who remain active in publishing maintain and establish more collaborations compared to their similar non-retracted counterparts. Nevertheless, retracted authors with less than a decade of publishing experience retain less senior, less productive and less impactful coauthors, and gain less senior coauthors post-retraction. Taken together, notwithstanding the indispensable role of retractions in upholding the integrity of the academic community, our findings shed light on the disproportionate impact that retractions impose on early-career authors., Comment: 49 pages, 13 figures, 19 tables

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space.The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment.

Facial recognition systems have made significant strides thanks to data-heavy deep learning models, but these models rely on large privacy-sensitive datasets. Further, many of these datasets lack diversity in terms of ethnicity and demographics, which can lead to biased models that can have serious societal and security implications. To address these issues, we propose a methodology that leverages the biased generative model StyleGAN2 to create demographically diverse images of synthetic individuals. The synthetic dataset is created using a novel evolutionary search algorithm that targets specific demographic groups. By training face recognition models with the resulting balanced dataset containing 50,000 identities per race (13.5 million images in total), we can improve their performance and minimize biases that might have been present in a model trained on a real dataset.

Artificial Intelligence (AI) has become a disruptive technology, promising to grant a significant economic and strategic advantage to nations that harness its power. China, with its recent push towards AI adoption, is challenging the U.S.'s position as the global leader in this field. Given AI's massive potential, as well as the fierce geopolitical tensions between China and the U.S., several recent policies have been put in place to discourage AI scientists from migrating to, or collaborating with, the other nation. Nevertheless, the extent of talent migration and cross-border collaboration are not fully understood. Here, we analyze a dataset of over 350,000 AI scientists and 5,000,000 AI papers. We find that since 2000, China and the U.S. have led the field in terms of impact, novelty, productivity, and workforce. Most AI scientists who move to China come from the U.S., and most who move to the U.S. come from China, highlighting a notable bidirectional talent migration. Moreover, the vast majority of those moving in either direction have Asian ancestry. Upon moving, those scientists continue to collaborate frequently with those in the origin country. Although the number of collaborations between the two countries has increased since the dawn of the millennium, such collaborations continue to be relatively rare. A matching experiment reveals that the two countries have always been more impactful when collaborating than when each works without the other. These findings suggest that instead of suppressing cross-border migration and collaboration between the two nations, the science could benefit from promoting such activities., Comment: 38 pages, 15 figures, 3 tables

Face swapping technology used to create "Deepfakes" has advanced significantly over the past few years and now enables us to create realistic facial manipulations. Current deep learning algorithms to detect deepfakes have shown promising results, however, they require large amounts of training data, and as we show they are biased towards a particular ethnicity. We propose a deepfake detection methodology that eliminates the need for any real data by making use of synthetically generated data using StyleGAN3. This not only performs at par with the traditional training methodology of using real data but it shows better generalization capabilities when finetuned with a small amount of real data. Furthermore, this also reduces biases created by facial image datasets that might have sparse data from particular ethnicities., Comment: IJCB 2022

With the rise of AI-enabled Real-Time Deepfakes (RTDFs), the integrity of online video interactions has become a growing concern. RTDFs have now made it feasible to replace an imposter's face with their victim in live video interactions. Such advancement in deepfakes also coaxes detection to rise to the same standard. However, existing deepfake detection techniques are asynchronous and hence ill-suited for RTDFs. To bridge this gap, we propose a challenge-response approach that establishes authenticity in live settings. We focus on talking-head style video interaction and present a taxonomy of challenges that specifically target inherent limitations of RTDF generation pipelines. We evaluate representative examples from the taxonomy by collecting a unique dataset comprising eight challenges, which consistently and visibly degrades the quality of state-of-the-art deepfake generators. These results are corroborated both by humans and a new automated scoring function, leading to 88.6% and 80.1% AUC, respectively. The findings underscore the promising potential of challenge-response systems for explainable and scalable real-time deepfake detection in practical scenarios. We provide access to data and code at \url{https://github.com/mittalgovind/GOTCHA-Deepfakes}., Comment: Accepted to IEEE Euro S&P 2024

This work expands on previous advancements in genetic fingerprint spoofing via the DeepMasterPrints and introduces Diversity and Novelty MasterPrints. This system uses quality diversity evolutionary algorithms to generate dictionaries of artificial prints with a focus on increasing coverage of users from the dataset. The Diversity MasterPrints focus on generating solution prints that match with users not covered by previously found prints, and the Novelty MasterPrints explicitly search for prints with more that are farther in user space than previous prints. Our multi-print search methodologies outperform the singular DeepMasterPrints in both coverage and generalization while maintaining quality of the fingerprint image output.

Active contour models have achieved prominent success in the area of image segmentation, allowing complex objects to be segmented from the background for further analysis. Existing models can be divided into region-based active contour models and edge-based active contour models. However, both models use direct image data to achieve segmentation and face many challenging problems in terms of the initial contour position, noise sensitivity, local minima and inefficiency owing to the in-homogeneity of image intensities. The saliency map of an image changes the image representation, making it more visual and meaningful. In this study, we propose a novel model that uses the advantages of a saliency map with local image information (LIF) and overcomes the drawbacks of previous models. The proposed model is driven by a saliency map of an image and the local image information to enhance the progress of the active contour models. In this model, the saliency map of an image is first computed to find the saliency driven local fitting energy. Then, the saliency-driven local fitting energy is combined with the LIF model, resulting in a final novel energy functional. This final energy functional is formulated through a level set formulation, and regulation terms are added to evolve the contour more precisely across the object boundaries. The quality of the proposed method was verified on different synthetic images, real images and publicly available datasets, including medical images. The image segmentation results, and quantitative comparisons confirmed the contour initialization independence, noise insensitivity, and superior segmentation accuracy of the proposed model in comparison to the other segmentation models.

This report presents a connection between the physical acoustics of an object and the topology of the space of echoes resulting from a circular synthetic aperture sonar (CSAS) collection of that object. A simple theoretical model is developed that yields a precise, yet qualitative, description of the space of echoes. This theoretical model is validated in simulation and with experimental data from a laboratory sonar system., Comment: 21 pages, 23 figures

In this paper, we propose dictionary attacks against speaker verification - a novel attack vector that aims to match a large fraction of speaker population by chance. We introduce a generic formulation of the attack that can be used with various speech representations and threat models. The attacker uses adversarial optimization to maximize raw similarity of speaker embeddings between a seed speech sample and a proxy population. The resulting master voice successfully matches a non-trivial fraction of people in an unknown population. Adversarial waveforms obtained with our approach can match on average 69% of females and 38% of males enrolled in the target system at a strict decision threshold calibrated to yield false alarm rate of 1%. By using the attack with a black-box voice cloning system, we obtain master voices that are effective in the most challenging conditions and transferable between speaker encoders. We also show that, combined with multiple attempts, this attack opens even more to serious issues on the security of these systems., Comment: Accepted in IEEE Transactions on Information Forensics and Security

Heading and position control system of ships has remained a challenging control problem. It is a nonlinear multiple input multiple output system. Moreover, the dynamics of the system vary with operating as well as environmental conditions. Conventionally, simple Proportional Integral Derivative controller is used which has well known limitations. Other conventional control techniques have also been investigated but they require an accurate mathematical model of a ship. Unfortunately, accuracy of mathematical models is very difficult to achieve. During the past few decades computational intelligence techniques such as artificial neural networks have been very successful when an accurate mathematical model is not available. Therefore, in this paper, an artificial neural network controller is proposed for heading and position control system. For simulation purposes, a mathematical model with four effective thrusters have been chosen to test the performance of the proposed controller. The final closed loop system has been analyzed and tested through simulation studies. The results are very encouraging., Comment: 6 pages

Can we leverage high-resolution information without the unsustainable quadratic complexity to input scale? We propose Traversal Network (TNet), a novel multi-scale hard-attention architecture, which traverses image scale-space in a top-down fashion, visiting only the most informative image regions along the way. TNet offers an adjustable trade-off between accuracy and complexity, by changing the number of attended image locations. We compare our model against hard-attention baselines on ImageNet, achieving higher accuracy with less resources (FLOPs, processing time and memory). We further test our model on fMoW dataset, where we process satellite images of size up to $896 \times 896$ px, getting up to $2.5$x faster processing compared to baselines operating on the same resolution, while achieving higher accuracy as well. TNet is modular, meaning that most classification models could be adopted as its backbone for feature extraction, making the reported performance gains orthogonal to benefits offered by existing optimized deep models. Finally, hard-attention guarantees a degree of interpretability to our model's predictions, without any extra cost beyond inference. Code is available at $\href{https://github.com/Tpap/TNet}{github.com/Tpap/TNet}$.

The mathematical interpretation of interventions for the mitigation of epidemics and pandemics in the literature often involves finding the optimal time to initiate an intervention and/or the use of infections to manage impact. Whilst these methods may work in theory, in order to implement they may require information which is likely not available whilst one is in the midst of an epidemic, or they may require impeccable data about infection levels in the community. In practice, testing and cases data is only as good as the policy of implementation and the compliance of the individuals, which means that understanding the levels of infections becomes difficult or complicated from the data that is provided. In this paper, we aim to develop a different approach to the mathematical modelling of interventions, not based on optimality, but based on demand and capacity of local authorities who have to deal with the epidemic on a day to day basis. In particular, we use data-driven modelling to calibrate an Susceptible Exposed Infectious Recovered-Died (SEIR-D) model to infer parameters that depict the dynamics of the epidemic in a region of the UK. We use the calibrated parameters for forecasting scenarios and understand, given a maximum capacity of hospital healthcare services, how the timing of interventions, severity of interventions, and conditions for the releasing of interventions affect the overall epidemic-picture.

Fact checking by professionals is viewed as a vital defense in the fight against misinformation.While fact checking is important and its impact has been significant, fact checks could have limited visibility and may not reach the intended audience, such as those deeply embedded in polarized communities. Concerned citizens (i.e., the crowd), who are users of the platforms where misinformation appears, can play a crucial role in disseminating fact-checking information and in countering the spread of misinformation. To explore if this is the case, we conduct a data-driven study of misinformation on the Twitter platform, focusing on tweets related to the COVID-19 pandemic, analyzing the spread of misinformation, professional fact checks, and the crowd response to popular misleading claims about COVID-19. In this work, we curate a dataset of false claims and statements that seek to challenge or refute them. We train a classifier to create a novel dataset of 155,468 COVID-19-related tweets, containing 33,237 false claims and 33,413 refuting arguments.Our findings show that professional fact-checking tweets have limited volume and reach. In contrast, we observe that the surge in misinformation tweets results in a quick response and a corresponding increase in tweets that refute such misinformation. More importantly, we find contrasting differences in the way the crowd refutes tweets, some tweets appear to be opinions, while others contain concrete evidence, such as a link to a reputed source. Our work provides insights into how misinformation is organically countered in social platforms by some of their users and the role they play in amplifying professional fact checks.These insights could lead to development of tools and mechanisms that can empower concerned citizens in combating misinformation. The code and data can be found in http://claws.cc.gatech.edu/covid_counter_misinformation.html., Comment: PrePrint - IEEE BigData 2020. The code and data can be found in http://claws.cc.gatech.edu/covid_counter_misinformation.html

Our voices are as distinctive as our faces and fingerprints. There is a spectrum of non-disjoint traits that make our voices unique and identifiable, such as the fundamental frequency, the intensity, and most interestingly the quality of the speech. Voice quality refers to the characteristic features of an individual's voice. Previous research has from time-to-time proven the ubiquity of voice quality in making different paralinguistic inferences. These inferences range from identifying personality traits, to health conditions and beyond. In this manuscript, we first map the paralinguistic voice qualifiers to their acoustic correlates in the light of the previous research and literature. We also determine the openSMILE correlates one could possibly use to measure those correlates. In the second part, we give a set of example paralinguistic inferences that can be made using different acoustic and perceptual voice quality features.

From conspiracy theories to fake cures and fake treatments, COVID-19 has become a hot-bed for the spread of misinformation online. It is more important than ever to identify methods to debunk and correct false information online. In this paper, we present a methodology and analyses to characterize the two competing COVID-19 misinformation communities online: (i) misinformed users or users who are actively posting misinformation, and (ii) informed users or users who are actively spreading true information, or calling out misinformation. The goals of this study are two-fold: (i) collecting a diverse set of annotated COVID-19 Twitter dataset that can be used by the research community to conduct meaningful analysis; and (ii) characterizing the two target communities in terms of their network structure, linguistic patterns, and their membership in other communities. Our analyses show that COVID-19 misinformed communities are denser, and more organized than informed communities, with a possibility of a high volume of the misinformation being part of disinformation campaigns. Our analyses also suggest that a large majority of misinformed users may be anti-vaxxers. Finally, our sociolinguistic analyses suggest that COVID-19 informed users tend to use more narratives than misinformed users., Comment: 9 pages, In Proceedings of The 5th International Workshop on Mining Actionable Insights from Social Networks (MAISoN 2020), co-located with CIKM

Public health practitioners and policy makers grapple with the challenge of devising effective message-based interventions for debunking public health misinformation in cyber communities. "Framing" and "personalization" of the message is one of the key features for devising a persuasive messaging strategy. For an effective health communication, it is imperative to focus on "preference-based framing" where the preferences of the target sub-community are taken into consideration. To achieve that, it is important to understand and hence characterize the target sub-communities in terms of their social interactions. In the context of health-related misinformation, vaccination remains to be the most prevalent topic of discord. Hence, in this paper, we conduct a sociolinguistic analysis of the two competing vaccination communities on Twitter: "pro-vaxxers" or individuals who believe in the effectiveness of vaccinations, and "anti-vaxxers" or individuals who are opposed to vaccinations. Our data analysis show significant linguistic variation between the two communities in terms of their usage of linguistic intensifiers, pronouns, and uncertainty words. Our network-level analysis show significant differences between the two communities in terms of their network density, echo-chamberness, and the EI index. We hypothesize that these sociolinguistic differences can be used as proxies to characterize and understand these communities to devise better message interventions., Comment: 11 pages, 4 tables, 1 figure, 1 algorithm, accepted to SBP-BRiMS 2020 -- International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation

We assess the variability of PRNU-based camera fingerprints with mismatched imaging pipelines (e.g., different camera ISP or digital darkroom software). We show that camera fingerprints exhibit non-negligible variations in this setup, which may lead to unexpected degradation of detection statistics in real-world use-cases. We tested 13 different pipelines, including standard digital darkroom software and recent neural-networks. We observed that correlation between fingerprints from mismatched pipelines drops on average to 0.38 and the PCE detection statistic drops by over 40%. The degradation in error rates is the strongest for small patches commonly used in photo manipulation detection, and when neural networks are used for photo development. At a fixed 0.5% FPR setting, the TPR drops by 17 ppt (percentage points) for 128 px and 256 px patches., Comment: 6 pages and 3 pages supplemental file

PRNU based camera recognition method is widely studied in the image forensic literature. In recent years, CNN based camera model recognition methods have been developed. These two methods also provide solutions to tamper localization problem. In this paper, we propose their combination via a Neural Network to achieve better small-scale tamper detection performance. According to the results, the fusion method performs better than underlying methods even under high JPEG compression. For forgeries as small as 100$\times$100 pixel size, the proposed method outperforms the state-of-the-art, which validates the usefulness of fusion for localization of small-size image forgeries. We believe the proposed approach is feasible for any tamper-detection pipeline using the PRNU based methodology., Comment: 13 pages

Given the ubiquity of handwritten documents in human transactions, Optical Character Recognition (OCR) of documents have invaluable practical worth. Optical character recognition is a science that enables to translate various types of documents or images into analyzable, editable and searchable data. During last decade, researchers have used artificial intelligence / machine learning tools to automatically analyze handwritten and printed documents in order to convert them into electronic format. The objective of this review paper is to summarize research that has been conducted on character recognition of handwritten documents and to provide research directions. In this Systematic Literature Review (SLR) we collected, synthesized and analyzed research articles on the topic of handwritten OCR (and closely related topics) which were published between year 2000 to 2018. We followed widely used electronic databases by following pre-defined review protocol. Articles were searched using keywords, forward reference searching and backward reference searching in order to search all the articles related to the topic. After carefully following study selection process 142 articles were selected for this SLR. This review article serves the purpose of presenting state of the art results and techniques on OCR and also provide research directions by highlighting research gaps.

Can vocal emotions be emulated? This question has been a recurrent concern of the speech community, and has also been vigorously investigated. It has been fueled further by its link to the issue of validity of acted emotion databases. Much of the speech and vocal emotion research has relied on acted emotion databases as valid proxies for studying natural emotions. To create models that generalize to natural settings, it is crucial to work with valid prototypes -- ones that can be assumed to reliably represent natural emotions. More concretely, it is important to study emulated emotions against natural emotions in terms of their physiological, and psychological concomitants. In this paper, we present an on-scale systematic study of the differences between natural and acted vocal emotions. We use a self-attention based emotion classification model to understand the phonetic bases of emotions by discovering the most 'attended' phonemes for each class of emotions. We then compare these attended-phonemes in their importance and distribution across acted and natural classes. Our tests show significant differences in the manner and choice of phonemes in acted and natural speech, concluding moderate to low validity and value in using acted speech databases for emotion classification tasks., Comment: 5 pages, 6 figures

Do men and women perceive emotions differently? Popular convictions place women as more emotionally perceptive than men. Empirical findings, however, remain inconclusive. Most prior studies focus on visual modalities. In addition, almost all of the studies are limited to experiments within controlled environments. Generalizability and scalability of these studies has not been sufficiently established. In this paper, we study the differences in perception of emotion between genders from speech data in the wild, annotated through crowdsourcing. While we limit ourselves to a single modality (i.e. speech), our framework is applicable to studies of emotion perception from all such loosely annotated data in general. Our paper addresses multiple serious challenges related to making statistically viable conclusions from crowdsourced data. Overall, the contributions of this paper are two fold: a reliable novel framework for perceptual studies from crowdsourced data; and the demonstration of statistically significant differences in speech-based emotion perception between genders.

Named Entity Recognition (NER) System aims to extract the existing information into the following categories such as: Persons Name, Organization, Location, Date and Time, Term, Designation and Short forms. Now, it is considered to be important aspect for many natural languages processing (NLP) tasks such as: information retrieval system, machine translation system, information extraction system and question answering. Even at a surface level, the understanding of the named entities involved in a document gives richer analytical framework and cross referencing. It has been used for different Arabic Script-Based languages like, Arabic, Persian and Urdu but, Sindhi could not come into being yet. This paper explains the problem of NER in the framework of Sindhi Language and provides relevant solution. The system is developed to tag ten different Named Entities. We have used Ruled based approach for NER system of Sindhi Language. For the training and testing, 936 words were used and calculated performance accuracy of 98.71%., Comment: 10 Pages, 3 figures

Learning technology was used as standalone software to install in a particular system, which needs to buy learning software of a particular subject. It was costly and difficult to search CD/DVD of the particular program in the market. Nowadays the trend of learning is changed and people are learning via the internet and it is known as Electronic Learning (E-learning). Several e-learning web applications are available which are providing more stuff about students and it fulfills requirements. The aim of this paper is to present a well structured, user-friendly framework with the web application for e-learning, which does not need any subscription. The experiment was conducted with 691 students and teachers, the result shows 91.98% of participants were satisfied with the proposed E-learning system.

Photo Response Non-Uniformity (PRNU) based camera attribution is an effective method to determine the source camera of visual media (an image or a video). To apply this method, images or videos need to be obtained from a camera to create a "camera fingerprint" which then can be compared against the PRNU of the query media whose origin is under question. The fingerprint extraction process can be time-consuming when a large number of video frames or images have to be denoised. This may need to be done when the individual images have been subjected to high compression or other geometric processing such as video stabilization. This paper investigates a simple, yet effective and efficient technique to create a camera fingerprint when so many still images need to be denoised. The technique utilizes Spatial Domain Averaged (SDA) frames. An SDA-frame is the arithmetic mean of multiple still images. When it is used for fingerprint extraction, the number of denoising operations can be significantly decreased with little or no performance loss. Experimental results show that the proposed method can work more than 50 times faster than conventional methods while providing similar matching results., Comment: 11 pages, 9 figures, submitted to IEEE Transactions on Information Forensics and Security journal

We present FiFTy, a modern file type identification tool for memory forensics and data carving. In contrast to previous approaches based on hand-crafted features, we design a compact neural network architecture, which uses a trainable embedding space, akin to successful natural language processing models. Our approach dispenses with explicit feature extraction which is a bottleneck in legacy systems. We evaluate the proposed method on a novel dataset with 75 file types - the most diverse and balanced dataset reported to date. FiFTy consistently outperforms all baselines in terms of speed, accuracy and individual misclassification rates. We achieved an average accuracy of 77.5% with processing speed of approx 38 sec/GB, which is better and more than an order of magnitude faster than the previous state-of-the-art tool - Sceadan (69% at 9 min/GB). Our tool and the corresponding dataset are available publicly online., Comment: Paper accepted for publication in the IEEE Transactions on Information Forensics and Security

Autonomous driving is expected to provide a range of far-reaching economic, environmental and safety benefits. In this study, we propose a fog computing based framework to assist autonomous driving. Our framework relies on overhead views from cameras and data streams from vehicle sensors to create a network of distributed digital twins, called an edge twin, on fog machines. The edge twin will be continuously updated with the locations of both autonomous and human-piloted vehicles on the road segments. The vehicle locations will be harvested from overhead cameras as well as location feeds from the vehicles themselves. Although the edge twin can make fair road space allocations from a global viewpoint, there is a communication cost (delay) in reaching it from the cameras and vehicular sensors. To address this, we introduce a machine learning forecaster as a part of the edge twin which is responsible for predicting the future location of vehicles. Lastly, we introduce a box algorithm that will use the forecasted values to create a hazard map for the road segment which would be used by the framework to suggest safe manoeuvres for the autonomous vehicles such as lane changes and accelerations. We present the complete fog computing framework for autonomous driving assist and evaluate key portions of the proposed framework using simulations based on a real-world dataset of vehicle position traces on a highway

For effective use of edge computing in an IoT application, we need to partition the application into tasks and map them into the cloud, fog (edge server), device levels such that the resources at the different levels are optimally used to meet the overall quality of service requirements. In this paper, we consider four concerns about application-to-fog mapping: task placement at different levels, data filtering to limit network loading, fog fail-over, and data consistency, and reacting to hotspots at the edge. We describe a programming language and middleware we created for edge computing that addresses the above four concerns. The language has a distributed-node programming model that allows programs to be written for a collection of nodes organized into a cloud, fog, device hierarchy. The paper describes the major design elements of the language and explains the prototype implementation. The unique distributed-node programming model embodied in the language enables new edge-oriented programming patterns that are highly suitable for cognitive or data-intensive edge computing workloads. The paper presents result from an initial evaluation of the language prototype and also a distributed shell and a smart parking app that were developed using the programming language., Comment: 22 pages, 9 figures, journal

Photo Response Non-Uniformity (PRNU) based source camera attribution is an effective method to determine the origin camera of visual media (an image or a video). However, given that modern devices, especially smartphones, capture images, and videos at different resolutions using the same sensor array, PRNU attribution can become ineffective as the camera fingerprint and query visual media can be misaligned. We examine different resizing techniques such as binning, line-skipping, cropping and scaling that cameras use to downsize the raw sensor image to different media. Taking such techniques into account, this paper studies the problem of source camera attribution. We define the notion of Ratio of Alignment, which is a measure of shared sensor elements among spatially corresponding pixels within two media objects resized with different techniques. We then compute the Ratio of Alignment between the different combinations of three common resizing methods under simplified conditions and experimentally validate our analysis. Based on the insights drawn from the different techniques used by cameras and the RoA analysis, the paper proposes an algorithm for matching the source of a video with an image and vice versa. We also present an efficient search method resulting in significantly improved performance in matching as well as computation time., Comment: 16 pages, 9 figures, 11 tables. "The paper is under consideration at Pattern Recognition Letters"

ENF (Electrical Network Frequency) instantaneously fluctuates around its nominal value (50/60 Hz) due to a continuous disparity between generated power and consumed power. Consequently, luminous intensity of a mains-powered light source varies depending on ENF fluctuations in the grid network. Variations in the luminance over time can be captured from video recordings and ENF can be estimated through content analysis of these recordings. In ENF based video forensics, it is critical to check whether a given video file is appropriate for this type of analysis. That is, if ENF signal is not present in a given video, it would be useless to apply ENF based forensic analysis. In this work, an ENF signal presence detection method is introduced for videos. The proposed method is based on multiple ENF signal estimations from steady superpixels, i.e. pixels that are most likely uniform in color, brightness, and texture, and intraclass similarity of the estimated signals. Subsequently, consistency among these estimates is then used to determine the presence or absence of an ENF signal in a given video. The proposed technique can operate on video clips as short as 2 minutes and is independent of the camera sensor type, i.e. CCD or CMOS.

ENF is a time-varying signal of the frequency of mains electricity in a power grid. It continuously fluctuates around a nominal value (50/60 Hz) due to changes in supply and demand of power over time. Depending on these ENF variations, the luminous intensity of a mains-powered light source also fluctuates. These fluctuations in luminance can be captured by video recordings. Accordingly, ENF can be estimated from such videos by analysis of steady content in the video scene. When videos are captured by using a rolling shutter sampling mechanism, as is done mostly with CMOS cameras, there is an idle period between successive frames. Consequently, a number of illumination samples of the scene are effectively lost due to the idle period. These missing samples affect ENF estimation, in the sense of the frequency shift caused and the power attenuation that results. This work develops an analytical model for videos captured using a rolling shutter mechanism. The model illustrates how the frequency of the main ENF harmonic varies depending on the idle period length, and how the power of the captured ENF attenuates as idle period increases. Based on this, a novel idle period estimation method for potential use in camera forensics that is able to operate independently of video frame rate is proposed. Finally, a novel time-of-recording verification approach based on use of multiple ENF components, idle period assumptions and interpolation of missing ENF samples is also proposed.