Search

This study aims to investigate the comprehensive characterization of information content in multimedia (videos), particularly on YouTube. The research presents a multi-method framework for characterizing multimedia content by clustering signals from various modalities, such as audio, video, and text. With a focus on South China Sea videos as a case study, this approach aims to enhance our understanding of online content, especially on YouTube. The dataset includes 160 videos, and our findings offer insights into content themes and patterns within different modalities of a video based on clusters. Text modality analysis revealed topical themes related to geopolitical countries, strategies, and global security, while video and audio modality analysis identified distinct patterns of signals related to diverse sets of videos, including news analysis/reporting, educational content, and interviews. Furthermore, our findings uncover instances of content repurposing within video clusters, which were identified using the barcode technique and audio similarity assessments. These findings indicate potential content amplification techniques. In conclusion, this study uniquely enhances our current understanding of multimedia content information based on modality clustering techniques., Comment: 14 pages, In the 4th International Conference on SMART MULTIMEDIA, 2024

Azure Cognitive Search (ACS) has emerged as a major contender in "Search as a Service" cloud products in recent years. However, one of the major challenges for ACS users is to improve the relevance of the search results for their specific usecases. In this paper, we propose a novel method to find the optimal ACS configuration that maximizes search relevance for a specific usecase (product search, document search...) The proposed solution improves key online marketplace metrics such as click through rates (CTR) by formulating the search relevance problem as hyperparameter tuning. We have observed significant improvements in real-world search call to action (CTA) rate in multiple marketplaces by introducing optimized weights generated from the proposed approach.

Technological advancement and its omnipresent connection have pushed humans past the boundaries and limitations of a computer screen, physical state, or geographical location. It has provided a depth of avenues that facilitate human-computer interaction that was once inconceivable such as audio and body language detection. Given the complex modularities of emotions, it becomes vital to study human-computer interaction, as it is the commencement of a thorough understanding of the emotional state of users and, in the context of social networks, the producers of multimodal information. This study first acknowledges the accuracy of classification found within multimodal emotion detection systems compared to unimodal solutions. Second, it explores the characterization of multimedia content produced based on their emotions and the coherence of emotion in different modalities by utilizing deep learning models to classify emotion across different modalities., Comment: 8 pages, Published in International Conference on Computers and Computation (COMPUTE 2022), November 03-04, 2022, San Francisco, United States

YouTube, a widely popular online platform, has transformed the dynamics of con-tent consumption and interaction for users worldwide. With its extensive range of content crea-tors and viewers, YouTube serves as a hub for video sharing, entertainment, and information dissemination. However, the exponential growth of users and their active engagement on the platform has raised concerns regarding suspicious commenter behaviors, particularly in the com-ment section. This paper presents a social network analysis-based methodology for detecting suspicious commenter mob-like behaviors among YouTube channels and the similarities therein. The method aims to characterize channels based on the level of such behavior and identify com-mon patterns across them. To evaluate the effectiveness of the proposed model, we conducted an analysis of 20 YouTube channels, consisting of 7,782 videos, 294,199 commenters, and 596,982 comments. These channels were specifically selected for propagating false views about the U.S. Military. The analysis revealed significant similarities among the channels, shedding light on the prevalence of suspicious commenter behavior. By understanding these similarities, we contribute to a better understanding of the dynamics of suspicious behavior on YouTube channels, which can inform strategies for addressing and mitigating such behavior.

YouTube is the second most visited website in the world and receives comments from millions of commenters daily. The comments section acts as a space for discussions among commenters, but it could also be a breeding ground for problematic behavior. In particular, the presence of suspicious commenters who engage in activities that deviate from the norms of constructive and respectful discourse can negatively impact the community and the quality of the online experience. This paper presents a social network analysis-based methodology for detecting commenter mobs on YouTube. These mobs of commenters collaborate to boost engagement on certain videos. The method provides a way to characterize channels based on the level of suspicious commenter behavior and detect coordination among channels. To evaluate our model, we analyzed 20 YouTube channels, 7,782 videos, 294,199 commenters, and 596,982 comments that propagated false views about the U.S. Military. The analysis concluded with evidence of commenter mob activities, possible coordinated suspicious behavior on the channels, and an explanation of the behavior of co-commenter communities.

Deliberation and communication within the national space have had numerous implications on how citizens online and offline perceive government. It has also impacted the relationship between opposition and incumbent governments in the Indo-Pacific region. Authoritarian regimes have historically had control over the dissemination of information, thereby controlling power and limiting challenges from citizens who are not comfortable with the status quo. Social media and blogs have allowed citizens of these countries to find a way to communicate, and the exchange of information continues to rise. The quest by both authoritarian and democratic regimes to control or influence the discussion in the public sphere has given rise to concepts like cybertroopers, congressional bloggers, and commentator bloggers, among others. Cybertroopers have become the de facto online soldiers of authoritarian regimes who must embrace democracy. While commentator and congressional bloggers have acted with different strategies, commentator bloggers educate online citizens with knowledgeable information to influence the citizens. Congressional bloggers are political officeholders who use blogging to communicate their positions on ongoing national issues. Therefore, this work has explored various concepts synonymous with the Indo-Pacific public sphere and how it shapes elections and democracy.

The emergence of toxic information on social networking sites, such as Twitter, Parler, and Reddit, has become a growing concern. Consequently, this study aims to assess the level of toxicity in COVID-19 discussions on Twitter, Parler, and Reddit. Using data analysis from January 1 through December 31, 2020, we examine the development of toxicity over time and compare the findings across the three platforms. The results indicate that Parler had lower toxicity levels than both Twitter and Reddit in discussions related to COVID-19. In contrast, Reddit showed the highest levels of toxicity, largely due to various anti-vaccine forums that spread misinformation about COVID-19 vaccines. Notably, our analysis of COVID-19 vaccination conversations on Twitter also revealed a significant presence of conspiracy theories among individuals with highly toxic attitudes. Our computational approach provides decision-makers with useful information about reducing the spread of toxicity within online communities. The study's findings highlight the importance of taking action to encourage more uplifting and productive online discourse across all platforms.

The proliferation of social media platforms like Twitter has heightened the consequences of the spread of misinformation. To understand and model the spread of misinformation, in this paper, we leveraged the SEIZ (Susceptible, Exposed, Infected, Skeptics) epidemiological model to describe the underlying process that delineates the spread of misinformation on Twitter. Compared to the other epidemiological models, this model produces broader results because it includes the additional Skeptics (Z) compartment, wherein a user may be exposed to an item of misinformation but not engage in any reaction to it, and the additional Exposed (E) compartment, wherein the user may need some time before deciding to spread a misinformation item. We analyzed misinformation regarding the unrest in Washington, D.C. in the month of March 2020 which was propagated by the use of the #DCblackout hashtag by different users across the U.S. on Twitter. Our analysis shows that misinformation can be modeled using the concept of epidemiology. To the best of our knowledge, this research is the first to attempt to apply the SEIZ epidemiological model to the spread of a specific item of misinformation, which is a category distinct from that of rumor, and a hoax on online social media platforms. Applying a mathematical model can help to understand the trends and dynamics of the spread of misinformation on Twitter and ultimately help to develop techniques to quickly identify and control it., Comment: This paper is accepted on the International Conference on Fake News, Social Media Manipulation and Misinformation 2021 (ICFNSMMM 2021)

Creating realistic styled spaces is a complex task, which involves design know-how for what furniture pieces go well together. Interior style follows abstract rules involving color, geometry and other visual elements. Following such rules, users manually select similar-style items from large repositories of 3D furniture models, a process which is both laborious and time-consuming. We propose a method for fast-tracking style-similarity tasks, by learning a furniture's style-compatibility from interior scene images. Such images contain more style information than images depicting single furniture. To understand style, we train a deep learning network on a classification task. Based on image embeddings extracted from our network, we measure stylistic compatibility of furniture. We demonstrate our method with several 3D model style-compatibility results, and with an interactive system for modeling style-consistent scenes., Comment: Accepted to Pacific Graphics 2020

We present a new meshing algorithm called guided and augmented meshing, GAMesh, which uses a mesh prior to generate a surface for the output points of a point network. By projecting the output points onto this prior and simplifying the resulting mesh, GAMesh ensures a surface with the same topology as the mesh prior but whose geometric fidelity is controlled by the point network. This makes GAMesh independent of both the density and distribution of the output points, a common artifact in traditional surface reconstruction algorithms. We show that such a separation of geometry from topology can have several advantages especially in single-view shape prediction, fair evaluation of point networks and reconstructing surfaces for networks which output sparse point clouds. We further show that by training point networks with GAMesh, we can directly optimize the vertex positions to generate adaptive meshes with arbitrary topologies., Comment: Accepted to 3DV 2020

Sharp features such as edges and corners play an important role in the perception of 3D models. In order to capture them better, we propose quadric loss, a point-surface loss function, which minimizes the quadric error between the reconstructed points and the input surface. Computation of Quadric loss is easy, efficient since the quadric matrices can be computed apriori, and is fully differentiable, making quadric loss suitable for training point and mesh based architectures. Through extensive experiments we show the merits and demerits of quadric loss. When combined with Chamfer loss, quadric loss achieves better reconstruction results as compared to any one of them or other point-surface loss functions., Comment: Accepted to BMVC 2019 for Oral Presentation

Modeling spillover effects from observational data is an important problem in economics, business, and other fields of research. % It helps us infer the causality between two seemingly unrelated set of events. For example, if consumer spending in the United States declines, it has spillover effects on economies that depend on the U.S. as their largest export market. In this paper, we aim to infer the causation that results in spillover effects between pairs of entities (or units), we call this effect as \textit{paired spillover}. To achieve this, we leverage the recent developments in variational inference and deep learning techniques to propose a generative model called Linked Causal Variational Autoencoder (LCVA). Similar to variational autoencoders (VAE), LCVA incorporates an encoder neural network to learn the latent attributes and a decoder network to reconstruct the inputs. However, unlike VAE, LCVA treats the \textit{latent attributes as confounders that are assumed to affect both the treatment and the outcome of units}. Specifically, given a pair of units $u$ and $\bar{u}$, their individual treatment and outcomes, the encoder network of LCVA samples the confounders by conditioning on the observed covariates of $u$, the treatments of both $u$ and $\bar{u}$ and the outcome of $u$. Once inferred, the latent attributes (or confounders) of $u$ captures the spillover effect of $\bar{u}$ on $u$. Using a network of users from job training dataset (LaLonde (1986)) and co-purchase dataset from Amazon e-commerce domain, we show that LCVA is significantly more robust than existing methods in capturing spillover effects., Comment: First two authors contributed equally, 4 pages, CIKM'18

Diabetic-induced peripheral neuropathy (DPN) is a diabetic late complication. The molecular mechanisms underlying the pathophysiology of nerve damage & sensory loss remain largely unclear. Recently, alterations in metabolic flux have gained attention a basis for organ damage in diabetes; however, peripheral sensory neurons have not been adequately analyzed. In the present study, we attempted to delineate the role of alteration of metabolic pathways in relation to nerve damage & sensory loss. We employed STZ-injected mouse model of type1 diabetes. To investigate the progression of DPN by behavioral measurements of sensitivity to thermal & mechanical stimuli and quantitative assessment of intraepidermal nerve fiber density. We employed a MS-based screen to address alterations in levels of metabolites in peripheral sciatic nerve (SN) & amino acids (AA) in serum over several months post-STZ administration. Although hyperglycemia & body weight changes occurred early, sensory loss & reduced intraepithelial branching of nociceptive nerves was only evident at 22 wks post-STZ. The longitudinal metabolites screen in SN demonstrated that mice at 12 and 22 wks post-STZ showed an early impairment the tricarboxylic acid. We found that levels of citric acid, ketoglutaric acid, succinic acid, fumaric acid & malic acid were observed to be significantly reduced in SN at 22 wks post-STZ. In addition, we also found the increase in levels of sorbitol & L-Lactate in SN from 12 wks post-STZ injection. AA screen in serum showed that the amino acids Val, Ile and Leu, increased more than 2-fold from 12 wks post-STZ. Similarly, the levels of Tyr, Asn, Ser, His, Ala, & Pro showed progressive increase. Our results indicate that the impaired TCA cycle metabolites in peripheral nerve is the primary cause of shunting metabolic substrate to compensatory pathways which leads to mitochondrial dysfunction & nerve damage., Comment: 17 pages, 3 figures

Diabetic peripheral neuropathy (DPN) is one of the most common diabetic complications. Mechanisms underlying nerve damage and sensory loss following metabolic dysfunction remain large unclear. Recently, hyperglycemia-induced mitochondrial dysfunction and the generation of ROS have gained attention as possible mechanisms of organ damage in diabetes. Hypoxia-inducible factor 1(HIF1a) is a key transcription factor activated by hypoxia, hyperglycemia, nitric oxide as well as ROS, suggesting a fundamental role in DPN susceptibility. Genetically-modified mutant mice, which conditionally lack HIF1a in peripheral sensory neurons (SNS-HIF1a-/-), were analyzed longitudinally up to 6 months in the streptozotocin (STZ) model of type1 diabetes. Behavioral measurements of sensitivity to thermal and mechanical stimuli, quantitative morphological analyses of intraepidermal nerve fiber density and measurements of reactive oxygen species (ROS) in sensory neurons in vivo were undertaken over several months post-STZ injections to delineate the role of HIF1a in DPN. Longitudinal behavioral and morphological analyses at 5, 13 and 24 wks post-STZ treatment revealed that SNS-HIF1a-/- developed stronger hyperglycemia-evoked losses of peripheral nociceptive sensory axons associated with stronger losses of mechano- and heat sensation with a faster onset than HIF1afl/fl mice. Mechanistically, these histomorphologic and behavioral differences were associated with significantly higher level of STZ-induced production of ROS in sensory neurons of SNS-HIF1a-/- mice as compared with HIF1afl/fl. Our results indicate that HIF1a is as an upstream modulator of ROS in peripheral sensory neurons and exerts a protective function in suppressing hyperglycemia-induced nerve damage by limiting ROS levels. HIF1a stabilization may be thus a new strategy target for limiting sensory loss, a debilitating late complication of diabetes.

Brain mapping research in most neuroanatomical laboratories relies on conventional processing techniques, which often introduce histological artifacts such as tissue tears and tissue loss. In this paper we present techniques and algorithms for automatic registration and 3D reconstruction of conventionally produced mouse brain slices in a standardized atlas space. This is achieved first by constructing a virtual 3D mouse brain model from annotated slices of Allen Reference Atlas (ARA). Virtual re-slicing of the reconstructed model generates ARA-based slice images corresponding to the microscopic images of histological brain sections. These image pairs are aligned using a geometric approach through contour images. Histological artifacts in the microscopic images are detected and removed using Constrained Delaunay Triangulation before performing global alignment. Finally, non-linear registration is performed by solving Laplace's equation with Dirichlet boundary conditions. Our methods provide significant improvements over previously reported registration techniques for the tested slices in 3D space, especially on slices with significant histological artifacts. Further, as an application we count the number of neurons in various anatomical regions using a dataset of 51 microscopic slices from a single mouse brain. This work represents a significant contribution to this subfield of neuroscience as it provides tools to neuroanatomist for analyzing and processing histological data., Comment: 14 pages, 11 figures