Search

Temporal action proposal generation (TAPG) aims to estimate temporal intervals of actions in untrimmed videos, which is a challenging yet plays an important role in many tasks of video analysis and understanding. Despite the great achievement in TAPG, most existing works ignore the human perception of interaction between agents and the surrounding environment by applying a deep learning model as a black-box to the untrimmed videos to extract video visual representation. Therefore, it is beneficial and potentially improve the performance of TAPG if we can capture these interactions between agents and the environment. In this paper, we propose a novel framework named Agent-Aware Boundary Network (ABN), which consists of two sub-networks (i) an Agent-Aware Representation Network to obtain both agent-agent and agents-environment relationships in the video representation, and (ii) a Boundary Generation Network to estimate the confidence score of temporal intervals. In the Agent-Aware Representation Network, the interactions between agents are expressed through local pathway, which operates at a local level to focus on the motions of agents whereas the overall perception of the surroundings are expressed through global pathway, which operates at a global level to perceive the effects of agents-environment. Comprehensive evaluations on 20-action THUMOS-14 and 200-action ActivityNet-1.3 datasets with different backbone networks (i.e C3D, SlowFast and Two-Stream) show that our proposed ABN robustly outperforms state-of-the-art methods regardless of the employed backbone network on TAPG. We further examine the proposal quality by leveraging proposals generated by our method onto temporal action detection (TAD) frameworks and evaluate their detection performances. The source code can be found in this URL https://github.com/vhvkhoa/TAPG-AgentEnvNetwork.git., Comment: Accepted in the journal of IEEE Access Vol. 9

Identifying stress level can provide valuable data for mental health analytics as well as labels for annotation systems. Although much research has been conducted into stress detection models using heart rate variability at a higher cost of data collection, there is a lack of research on the potential of using low-resolution Electrodermal Activity (EDA) signals from consumer-grade wearable devices to identify stress patterns. In this paper, we concentrate on performing statistical analyses on the stress detection capability of two popular approaches of training stress detection models with stress-related biometric signals: user-dependent and user-independent models. Our research manages to show that user-dependent models are statistically more accurate for stress detection. In terms of effectiveness assessment, the balanced accuracy (BA) metric is employed to evaluate the capability of distinguishing stress and non-stress conditions of the models trained on either low-resolution or high-resolution Electrodermal Activity (EDA) signals. The results from the experiment show that training the model with (comparatively low-cost) low-resolution EDA signal does not affect the stress detection accuracy of the model significantly compared to using a high-resolution EDA signal. Our research results demonstrate the potential of attaching the user-dependent stress detection model trained on personal low-resolution EDA signal recorded to collect data in daily life to provide users with personal stress level insight and analysis.

Nowadays, with the rapid development of technology and an energetic lifestyle, people tend to use electronic devices to capture moments. Thus, the need to have a delightful selfie is increasing. This work introduces a Beauty Moment Rendering via Face Happiness Scoring framework, which aims to generate a key-frame that summarizes a short selfie video. Given a set of consecutive photos as input, our approach selects a single image and renders a face for each person at their happiest moment. In order to select a suitable key-frame, we decide to combine the eyes and emotion information as our aggregated score. First, the method detects and tracks the faces in the video frames. Then, the face that has the highest score of each person will be rendered on the result key-frame with the eye gazes being redirected based on a generative model. To the best of our knowledge, we are among the first to address the problem, which comprises video summarization and face beautification.

Cross-data analytics and retrieval have gained significant improvement recently. People can now extract more data insights precisely and quickly towards having many excellent applications serving human lives. Since people create multimedia and other types of data that reflect the diverse perspectives of human lives, these data are just pieces of the puzzle of the world's pictures. Hence, it is necessary to assembly all these pieces towards having a better solution for human-centered problems. Hence, the workshop welcomes those who work with multimedia and others and come from diverse research domains and disciplines to work on intelligent cross-data analytics and retrieval to bring a smart, sustainable society to human beings. The research domain can vary from well-being, disaster prevention and mitigation, mobility to food computing, to name a few.

In this paper, we present the interactive lifelog retrieval engine developed for the LSC'21 comparative benchmarking challenge. The LifeSeeker 3.0 interactive lifelog retrieval engine is an enhanced version of our previous system participating in LSC'20 - LifeSeeker 2.0. The system is developed by both Dublin City University and the Ho Chi Minh City University of Science. The implementation of LifeSeeker 3.0 focuses on searching and filtering by text query using a weighted Bag-of-Words model with visual concept augmentation and three weighted vocabularies. The visual similarity search is improved using a bag of local convolutional features; while improving the previous version's performance, enhancing query processing time, result displaying, and browsing support.

With a huge collection of photos and video clips, it is essential to provide an efficient and easy-to-use system for users to retrieve moments of interest with a wide variation of query types. This motivates us to develop and upgrade our flexible interactive retrieval system for visual lifelog exploration. In this paper, we briefly introduce version 2 of our system with the following main features. Our system supports multiple modalities for interaction and query processing, including visual query by meta-data, text query and visual information matching based on a joint embedding model, scene clustering based on visual and location information, flexible temporal event navigation, and query expansion with visual examples. With the flexibility in system architecture, we expect our system can easily integrate new modules to enhance its functionalities.

Camouflaged objects attempt to conceal their texture into the background and discriminating them from the background is hard even for human beings. The main objective of this paper is to explore the camouflaged object segmentation problem, namely, segmenting the camouflaged object(s) for a given image. This problem has not been well studied in spite of a wide range of potential applications including the preservation of wild animals and the discovery of new species, surveillance systems, search-and-rescue missions in the event of natural disasters such as earthquakes, floods or hurricanes. This paper addresses a new challenging problem of camouflaged object segmentation. To address this problem, we provide a new image dataset of camouflaged objects for benchmarking purposes. In addition, we propose a general end-to-end network, called the Anabranch Network, that leverages both classification and segmentation tasks. Different from existing networks for segmentation, our proposed network possesses the second branch for classification to predict the probability of containing camouflaged object(s) in an image, which is then fused into the main branch for segmentation to boost up the segmentation accuracy. Extensive experiments conducted on the newly built dataset demonstrate the effectiveness of our network using various fully convolutional networks. \url{https://sites.google.com/view/ltnghia/research/camo}, Comment: Published in CVIU 2019. Project page: https://sites.google.com/view/ltnghia/research/camo

Document vectorization with an appropriate encoding scheme is an essential component in various document processing tasks, including text document classification, retrieval, or generation. Training a dedicated document in a specific domain may require large enough data and sufficient resource. This motivates us to propose a novel document representation scheme with two main components. First, we train TD2V, a generic pre-trained document embedding for English documents from more than one million tweets in Twitter. Second, we propose a domain adaptation process with adversarial training to adapt TD2V to different domains. To classify a document, we use the rank list of its similar documents using query expansion techniques, either Average Query Expansion or Discriminative Query Expansion. Experiments on datasets from different online sources show that by using TD2V only, our method can classify documents with better accuracy than existing methods. By applying adversarial adaptation process, we can further boost and achieve the accuracy on BBC, BBCSport, Amazon4, 20NewsGroup datasets. We also evaluate our method on a specific domain of sensitivity classification and achieve the accuracy of higher than $$95\%$$ even with a short text fragment having 1024 characters on 5 datasets: Snowden, Mormon, Dyncorp, TM, and Enron.

Most popular systems for object instance search are based on the bag-of-visual-word model. The inherent weaknesses of this standard model such as quantization error, unstructured representation, burstiness phenomenon are to some extent solved. However, it has a serious problem of searching small objects on a database with cluttered background. In many situations, even the irrelevant objects which share the same texture or shape with a query object get higher score than relevant ones. To overcome this problem, we propose a novel fusion method to significantly boost the accuracy of instance search systems. Firstly, we use the state-of-the-art object detector with denser feature for finding object bounding box and similarity score. Secondly, to exploit the spatial relationship of each visual word with an object proposal, a detected area that might contain a query object, we define three categories of visual word pairs, i.e., discriminative, weak relevant, and context inferred ones. Finally, we propose a new re-ranking scheme with three weighting functions corresponding to the three categories of visual word pairs to compute the final similarity score between a query topic and a video shot. To illustrate the efficiency of the proposed method, we conduct experiments on datasets which have a wide variety of types of query objects. Experimental results on TRECVID Instance Search datasets (INS2013 and INS2014) show the superiority of our proposed method over the state-of-the-art approaches.

Flow-based generative models have recently become one of the most efficient approaches to model data generation. Indeed, they are constructed with a sequence of invertible and tractable transformations. Glow first introduced a simple type of generative flow using an invertible 1×1 convolution. However, the 1×1 convolution suffers from limited flexibility compared to the standard convolutions. In this paper, we propose a novel invertible n×n convolution approach that overcomes the limitations of the invertible 1×1 convolution. In addition, our proposed network is not only tractable and invertible but also uses fewer parameters than standard convolutions. The experiments on CIFAR-10, ImageNet and Celeb-HQ datasets, have shown that our invertible n×n convolution helps to improve the performance of generative models significantly.

Temporal action proposal generation is an essential and challenging task that aims at localizing temporal intervals containing human actions in untrimmed videos. Most of existing approaches are unable to follow the human cognitive process of understanding the video context due to lack of attention mechanism to express the concept of an action or an agent who performs the action or the interaction between the agent and the environment. Based on the action definition that a human, known as an agent, interacts with the environment and performs an action that affects the environment, we propose a contextual Agent-Environment Network. Our proposed contextual AEN involves (i) agent pathway, operating at a local level to tell about which humans/agents are acting and (ii) environment pathway operating at a global level to tell about how the agents interact with the environment. Comprehensive evaluations on 20-action THUMOS-14 and 200-action ActivityNet-1.3 datasets with different backbone networks, i.e C3D and SlowFast, show that our method robustly exhibits outperformance against state-of-the-art methods regardless of the employed backbone network., Accepted in ICASSP 2021

Language prior plays an important role in the way humans detect and recognize text in the wild. Current scene text recognition methods do use lexicons to improve recognition performance, but their naive approach of casting the output into a dictionary word based purely on the edit distance has many limitations. In this paper, we present a novel approach to incorporate a dictionary in both the training and inference stage of a scene text recognition system. We use the dictionary to generate a list of possible outcomes and find the one that is most compatible with the visual appearance of the text. The proposed method leads to a robust scene text recognition model, which is better at handling ambiguous cases encountered in the wild, and improves the overall performance of state-of-the-art scene text spotting frameworks. Our work suggests that incorporating language prior is a potential approach to advance scene text detection and recognition methods. Besides, we contribute VinText, a challenging scene text dataset for Vietnamese, where some characters are equivocal in the visual form due to accent symbols. This dataset will serve as a challenging benchmark for measuring the applicability and robustness of scene text detection and recognition algorithms. Code and dataset are available at https://github.com/VinAIResearch/dict-guided.

Multi-Camera Multiple Object Tracking (MC-MOT) is a significant computer vision problem due to its emerging applicability in several real-world applications. Despite a large number of existing works, solving the data association problem in any MC-MOT pipeline is arguably one of the most challenging tasks. Developing a robust MC-MOT system, however, is still highly challenging due to many practical issues such as inconsistent lighting conditions, varying object movement patterns, or the trajectory occlusions of the objects between the cameras. To address these problems, this work, therefore, proposes a new Dynamic Graph Model with Link Prediction (DyGLIP) approach to solve the data association task. Compared to existing methods, our new model offers several advantages, including better feature representations and the ability to recover from lost tracks during camera transitions. Moreover, our model works gracefully regardless of the overlapping ratios between the cameras. Experimental results show that we outperform existing MC-MOT algorithms by a large margin on several practical datasets. Notably, our model works favorably on online settings but can be extended to an incremental approach for large-scale datasets., accepted at CVPR 2021

Traffic event retrieval is one of the important tasks for intelligent traffic system management. To find accurate candidate events in traffic videos corresponding to a specific text query, it is necessary to understand the text query’s attributes, represent the visual and motion attributes of vehicles in videos, and measure the similarity between them. Thus we propose a promising method for vehicle event retrieval from a natural-language-based specification. We utilize both appearance and motion attributes of a vehicle and adapt the COOT model to evaluate the semantic relationship between a query and a video track. Experiments with the test dataset of Track 5 in AI City Challenge 2021 show that our method is among the top 6 with a score of 0.1560.

This paper pushes the envelope on decomposing camouflaged regions in an image into meaningful components, namely, camouflaged instances. To promote the new task of camouflaged instance segmentation of in-the-wild images, we introduce a dataset, dubbed CAMO++, that extends our preliminary CAMO dataset (camouflaged object segmentation) in terms of quantity and diversity. The new dataset substantially increases the number of images with hierarchical pixel-wise ground truths. We also provide a benchmark suite for the task of camouflaged instance segmentation. In particular, we present an extensive evaluation of state-of-the-art instance segmentation methods on our newly constructed CAMO++ dataset in various scenarios. We also present a camouflage fusion learning (CFL) framework for camouflaged instance segmentation to further improve the performance of state-of-the-art methods. The dataset, model, evaluation suite, and benchmark will be made publicly available on our project page: https://sites.google.com/view/ltnghia/research/camo_plus_plus, TIP acceptance. Project page: https://sites.google.com/view/ltnghia/research/camo_plus_plus

It is necessary to develop an efficient and secure mechanism to verify customers digitally for various online transactions. Integrating biometric solutions into the online user registration and verification processes is a promising trend for electronic Know Your Customer (eKYC) systems. However, Deepfake or face manipulation techniques may become a threat for eKYC with face authentication. In this paper, we introduce this potential attack of Deepfake on eKYC by swapping and manipulating faces between source and target faces. We then propose to augment the security for current eKYC systems with Deepfake detection. We conduct the experiments on the 10K video clips in the private test of Deepfake Detection Challenge 2020, and our method, following the Capsule-forensics approach, achieves the Logloss score of 0.5189, among the top 6% best results among the 2114 teams worldwide. This result demonstrates that our deepfake detection algorithm can be a promising method to provide extra protection for eKYC solutions with face registration and authentication.

In everyday life, authentication is an indispensable process of human activities. Bio-metric authentication system is one of the effective solutions, because it uses human-based features, instead of other traditional features, such as pin, password, etc. However, to apply a face authentication system in practical applications, we need to ensure that the system must not try to recognize the face of an unknown person into known categories, meaning we need to reject faces of unknown people in our application. In this paper, we present the limitations of recent Deep Learning based methods in Face Recognition tasks. We then propose two methods helping Face Recognition system have the ability to reject faces from unknown people by using Open-Set concepts. We conduct the experiments on a subset of CASIA-WebFace dataset, with a train set that includes 7000 images of 100 known people and a test set that includes both known and unknown people. Without rejecting unknown faces, the regular face recognition, i.e. the baseline method, yields the accuracy of only 45.9%, as the method tries to classify all face photos into known classes. Our proposed methods, which are combined deep network of Facenet system with recent Open Set methods, are called Learning Placeholder on Facenet (P-Facenet) and Facenet with OpenMax (O-Facenet). They achieve the accuracy of 83.6% and 88.5% respectively. This is a potential approach for authentication with face recognition to decrease the error rate of the model when recognizing faces of unknown people in the wild.

In the context of modern Android application development, security issues and secure programming are considered unignorable aspects to ensure the safety of Android applications while still guarantee the development speed. The lack of attention to security factors in the software development process or the delay of traditional security assurance methods are the main causes of unsafe Android software. Those unsafe Android applications contain many vulnerabilities and a high risk of leaking user information, especially since Android applications are rapidly developed and published. Developers must adhere to a secure development process to counter Android application risks to avoid data leakage or access control flaws. Security has to be integrated throughout the application development process to secure the software development life cycle. This paper presents two main research contributions: summarizing common security issues in Android applications and developing a plug-in for Android Studio to support secure programming, 9Fix. The low-time-cost 9Fix plug-in can inspect your vulnerable code and instantly suggest an alternative secure code for developers in programming time that helps to improve the security and instruct the developers on how to write a secure code. Moreover, the developers can add their own security rules to 9Fix so 9Fix can adapt smoothly in a specific situation. We also demonstrate the effectiveness and the convenience based on the student feedback by experimenting with the 9Fix plugin.

Polyps detection plays an important role in colonoscopy, cancer diagnosis, and early treatment. Many efforts have been made to improve the encoder-decoder framework using the global feature with an attention mechanism to enhance local features, helping to effectively segment diversity polyps. However, using only global information derived from the last encoder block leads to the loss of regional information from intermediate layers. Furthermore, defining the boundaries of some polyps is challenging because there is visual interference between the benign region and the polyps at the border. To address these problems, we propose two novel modules: the Cascading Context module (CCM) and the Attention Balance module (BAM), aiming to build an effective polyp segmentation model. Specifically, CCM combines the extracted regional information of the current layer and the lower layer, then pours it into the upper layer - fusing regional and global information analogous to a waterfall pattern. The BAM uses the prediction output of the adjacent lower layer as a guide map to implement the attention mechanism for the three regions separately: the background, polyp, and boundary curve. BAM enhances local context information when deriving features from the encoder block. Our proposed approach is evaluated on three benchmark datasets with six evaluation metrics for segmentation quality and gives competitive results compared to other advanced methods, for both accuracy and efficiency. Code is available at https://github.com/ntcongvn/CCBANet.

Gesture recognition is a fundamental tool to enable novel interaction paradigms in a variety of application scenarios like Mixed Reality environments, touchless public kiosks, entertainment systems, and more. Recognition of hand gestures can be nowadays performed directly from the stream of hand skeletons estimated by software provided by low-cost trackers (Ultraleap) and MR headsets (Hololens, Oculus Quest) or by video processing software modules (e.g. Google Mediapipe). Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario for the recognition of a wide set of heterogeneous gestures, as many benchmarks do not test online recognition and use limited dictionaries. This motivated the proposal of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild. For this contest, we created a novel dataset with heterogeneous gestures featuring different types and duration. These gestures have to be found inside sequences in an online recognition scenario. This paper presents the result of the contest, showing the performances of the techniques proposed by four research groups on the challenging task compared with a simple baseline method.

3D scene shape retrieval is a brand new but important research direction in content-based 3D shape retrieval. To promote this research area, two Shape Retrieval Contest (SHREC) tracks on 2D scene sketch-based and image-based 3D scene model retrieval have been organized by us in 2018 and 2019, respectively. In 2018, we built the first benchmark for each track which contains 2D and 3D scene data for ten (10) categories, while they share the same 3D scene target dataset. Four and five distinct 3D scene shape retrieval methods have competed with each other in these two contests, respectively. In 2019, to measure and compare the scalability performance of the participating and other promising Query-by-Sketch or Query-by-Image 3D scene shape retrieval methods, we built a much larger extended benchmark for each type of retrieval which has thirty (30) classes and organized two extended tracks. Again, two and three different 3D scene shape retrieval methods have contended in these two tracks, separately. To solicit state-of-the-art approaches, we perform a comprehensive comparison of all the above methods and an additional new retrieval methods by evaluating them on the two benchmarks. The benchmarks, evaluation results and tools are publicly available at our track websites (Yuan et al., 2019 [1]; Abdul-Rashid et al., 2019 [2]; Yuan et al., 2019 [3]; Abdul-Rashid et al., 2019 [4]), while code for the evaluated methods are also available: http://github.com/3DSceneRetrieval .

Record companies invest billions of dollars in new talent around the globe each year. Gaining insight into what actually makes a hit song would provide tremendous benefits for the music industry. In this research, we tackle this question by focusing on predicting rank of hit songs in the next 6 months. Our dataset is used in ZALO AI CHALLENGE 2019 in Hit Song Prediction problem including not only songs but also its information such as composer, artist name, released date, etc. Because of that, while most previous work formulates hit song prediction as a regression or classification problem, we present in this paper how to apply Gradient Boosting technique to treat it as a ranking problem. The resulting best model has a good performance when predicting whether a song is a top 10 dance hit versus a lower listed position with 1.48815 Root Mean Square Error - our result dominates most of the solution in this competition (better than 3rd ranked solution of 87 in total). Moreover, it is possible to further improve by extracting chords, tones and more information from each song to obtain the highlights of songs and by using linguistics model to offer high-level features of metadata.

Lifelog can provide useful insights of our daily activities. It is essential to provide a flexible way for users to retrieve certain events or moments of interest, corresponding to a wide variation of query types. This motivates us to develop FIRST, a Flexible Interactive Retrieval SysTem, to help users to combine or integrate various query components in a flexible manner to handle different query scenarios, such as visual clustering data based on color histogram, visual similarity, GPS location, or scene attributes. We also employ personalized concept detection and image captioning to enhance image understanding from visual lifelog data, and develop an autoencoder-like approach for query text and image feature mapping. Furthermore, we refine the user interface of the retrieval system to better assist users in query expansion and verifying sequential events in a flexible temporal resolution to control the navigation speed through sequences of images.

It is essential to develop efficient methods to detect abnormal events, such as car-crashes or stalled vehicles, from surveillance cameras to provide in-time help. This motivates us to propose a novel method to detect traffic accidents in traffic videos. To tackle the problem where anomalies only occupy a small amount of data, we propose a semi-supervised method using Generative Adversarial Network trained on regular sequences to predict future frames. Our key idea is to model the ordinary world with a generative model, then compare a predicted frame with the real next frame to determine if an abnormal event occurs. We also propose a new idea of encoding motion descriptors and scaled intensity loss function to optimize GAN for fast-moving objects. Experiments on the Traffic Anomaly Detection dataset of AI City Challenge 2019 show that our method achieves the top 3 results with F1 score 0.9412 and RMSE 4.8088, and S3 score 0.9261. Our method can be applied to different related applications of anomaly and outlier detection in videos.

Traffic flow analysis is essential for intelligent transportation systems. In this paper, we introduce our Intelligent Traffic Analysis Software Kit (iTASK) to tackle three challenging problems: vehicle flow counting, vehicle re-identification, and abnormal event detection. For the first problem, we propose to real-time track vehicles moving along the desired direction in corresponding motion-of-interests (MOIs). For the second problem, we consider each vehicle as a document with multiple semantic words (i.e., vehicle attributes) and transform the given problem to classical document retrieval. For the last problem, we propose to forward and backward refine anomaly detection using GAN-based future prediction and backward tracking completely stalled vehicle or sudden-change direction, respectively. Experiments on the datasets of traffic flow analysis from AI City Challenge 2020 show our competitive results, namely, S1 score of 0.8297 for vehicle flow counting in Track 1, mAP score of 0.3882 for vehicle re-identification in Track 2, and S4 score of 0.9059 for anomaly detection in Track 4. All data and source code are publicly available on our project page. 1

Recognition across domains has recently become an active topic in the research community. However, it has been largely overlooked in the problem of recognition in new unseen domains. Under this condition, the delivered deep network models are unable to be updated, adapted, or fine-tuned. Therefore, recent deep learning techniques, such as domain adaptation, feature transferring, and fine-tuning, cannot be applied. This paper presents a novel approach to the problem of domain generalization in the context of deep learning. The proposed method1 is evaluated on different datasets in various problems, i.e. (i) digit recognition on MNIST, SVHN, and MNIST-M, (ii) face recognition on Extended Yale-B, CMU-PIE and CMU-MPIE, and (iii) pedestrian recognition on RGB and Thermal image datasets. The experimental results show that our proposed method consistently improves performance accuracy. It can also be easily incorporated with any other CNN frameworks within an end-to-end deep network design for object detection and recognition problems to improve their performance.

The Lifelog Search Challenge (LSC) is an international content retrieval competition that evaluates search for personal lifelog data. At the LSC, content-based search is performed over a multi-modal dataset, continuously recorded by a lifelogger over 27 days, consisting of multimedia content, biometric data, human activity data, and information activities data. In this work, we report on the first LSC that took place in Yokohama, Japan in 2018 as a special workshop at ACM International Conference on Multimedia Retrieval 2018 (ICMR 2018). We describe the general idea of this challenge, summarise the participating search systems as well as the evaluation procedure, and analyse the search performance of the teams in various aspects. We try to identify reasons why some systems performed better than others and provide an outlook as well as open issues for upcoming iterations of the challenge.

This paper presents an overview of the ImageCLEF 2020 lab that was organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2020. ImageCLEF is an ongoing evaluation initiative (first run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks, i.e., caption analysis, tuberculosis prediction, and medical visual question answering and question generation, (ii) a lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (iii) a coral task about segmenting and labeling collections of coral reef images, and (iv) a new Internet task addressing the problems of identifying hand-drawn user interface components. Despite the current pandemic situation, the benchmark campaign received a strong participation with over 40 groups submitting more than 295 runs.

This paper presents the methods that have participated in the SHREC’20 contest on retrieval of surface patches with similar geometric reliefs and the analysis of their performance over the benchmark created for this challenge. The goal of the context is to verify the possibility of retrieving 3D models only based on the reliefs that are present on their surface and to compare methods that are suitable for this task. This problem is related to many real world applications, such as the classification of cultural heritage goods or the analysis of different materials. To address this challenge, it is necessary to characterize the local ”geometric pattern” information, possibly forgetting model size and bending. Seven groups participated in this contest and twenty runs were submitted for evaluation. The performances of the methods reveal that good results are achieved with a number of techniques that use different approaches.

Developing infrastructure and intelligent utilities for smart cities is an important trend in the world as well as in Vietnam. Thus, it is important to assist developers in building services for open data and smart city utilities. This motivates our proposal to develop a flexible platform with useful components, which can be integrated to develop these solutions quickly, to listen and analyze data from different social media sources with the diversification of data types, to provide open data providers in smart cities. Our method focuses on the ability to flexibly integrate artificial intelligence applications into the system to be able to both analyze effectively social events and serve smart cities in creating open data providers. We do not develop a particular system, but we create a platform, including different components, which are easy to be extended and integrated to create specific applications. To evaluate our platform, we develop four systems, including a face recognition system for celebrity recognition in news videos, an object detection system for brand logo recognition, a video highlighting system for summarizing football matches, and a text analysis system serving for keyword occurrences and emotional text analysis for admissions of universities. In these systems, we have collected and analyzed nearly 1000 videos from CNN, CBSN, FIFATV channels on YouTube, thousands of posts from admission pages of universities on Facebook. Each system gives a unique meaning to each specific situation for open data providers in smart cities.

Chest X-ray examination plays an important role in lung disease detection. The more accuracy of this task, the more experienced radiologists are required. After ChestX-ray14 dataset containing over 100,000 frontal-view X-ray images of 14 diseases was released, several models were proposed with high accuracy. In this paper, we develop a work flow for lung disease diagnosis in chest X-ray images, which can improve the average AUROC of the state-of-the-art model from 0.8414 to 0.8445. We apply image preprocessing steps before feeding to the 14 diseases detection model. Our project includes three models: the first one is DenseNet-121 to predict whether a processed image has a better result, a convolutional auto-encoder model for bone shadow exclusion is the second one, and the last is the original CheXNet., Comment: KSE 2018 long paper

The previous version of our retrieval system has shown some significant results in some retrieval tasks such as Lifelog’s moment retrieval tasks. In this paper, we adapt our platform to the Video Browser Showdown’s KIS and AVS tasks and present how our system performs in video search tasks. In addition to the smart features in our retrieval system that take advantage of the provided analysis data, we enhance the data with object color detection by employing Mask R-CNN and clustering. In this version of our search system, we try to extract the location information of the entities appearing in the videos and aim to exploit the spatial relationship between these entities. We also focus on designing efficient user interaction and a high-performance way to transfer data in the system to minimize the retrieval time.

Inspired by recent advances in computer vision and deep learning, we propose new enhancements to tackle problems appearing in endoscopic image analysis, especially abnormality finding and anatomical landmark detection. In details, a combination of Residual Neural Network and Faster R-CNN are jointly applied in order to take all of their advantages and improve the overall performance. Nevertheless, novel data augmentation is designed and adapted to corresponding domains. Our approaches prove their competitive results in term of not only the accuracy but also the inference time in Medico: The 2018 Multimedia for Medicine Task and The Biomedia ACM MM Grand Challenge 2019. These results show the great potential of the collaborating between deep learning models and data augmentation in medical image analysis applications. Especially, more than 4900 bounding boxes localizing the symptom of some classes from KVASIR dataset that we annotated and used in this project are shared online for future research.

SharkTank is a television show where start-ups pitch their idea to a panel of five investors (sharks) in hopes of striking a deal in the form of equity or royalties for money and other business perks. Since its inception, SharkTank has been a center of discussion and analysis for fans, statisticians, and business people alike in hopes of cracking the code to the start-up world and figuring out the formula for the next big ‘thing’. Most of these discussions and analyses have come in the form of blogs, articles, and academic research. However, there has been a lack of complete datasets and application of computational models for further analysis. In this paper, we investigate factors that play into the SharkTank deal. To this end, we first collect a new dataset, SharkTank Deal Dataset (STDD), by combining data from multiple public sources. The dataset includes descriptive features of each start-up such as product category, team composition, valuation, equity offering, specific sharks that appear on that episode, and state origin. For the computational model, we propose a new computational model to predict whether a start-up strikes a deal with a shark. We conduct experiments to demonstrate the superiority of our model over the baselines.

Lifelogging is a new trend of research in which we collect and analyze people's daily activities to gain insight into people daily basis and enhance people's life quality. This leads to many research in retrieval systems to be developed for lifelogging analysis. One of the most important factors of a successful lifelogging retrieval system is to enhance the user's ability to perform any queries, even arbitrary ones. This motivates our proposal of a retrieval system in which we focus on the two main properties. The first property is the ability to accurately annotate the lifelog dataset with metadata that is meaningful for the retrieving process. The second property focuses on a powerful user interface that supports a novice user to perform the retrieval efficiently.

Despite recent advances in deep learning, the crucial task of nuclear segmentation for computational pathology remains challenging. Recently, deep learning, and specifically U-Nets, have shown significant improvements for this task, but there is still room for improvement by further enhancing the design and training of U-Nets for nuclear segmentation. Specifically, we consider enforcing rotation equivariance in the network, the placement of residual blocks, and applying novel data augmentation designed specifically for histopathology images, and show the relative improvement and merit of each. Incorporating all of these enhancements in the design and training of a U-Net yields significantly improved segmentation results while still maintaining a speed of inference that is sufficient for real-world applications, in particular, analyzing whole-slide images (WSIs). Code for our enhanced U-Net is available at https://github.com/thatvinhton/G-U-Net.

With non-stop development of e-commerce and different internet-based applications’ demands, service providers include many physical servers scattering all over the world. If some users would like to use various services, they have to repeatedly register. Therefore, they must remember all these information such as, user-names or passwords. To solve this problem, many authentication schemes in multi-server environment are proposed. Furthermore, to prevent the adversary from keeping track of another user when he or she logins, many schemes apply dynamic identity, but they have some limitations with popular kinds of attacks such as, replay attack, impersonation attack, or man-in-the-middle attack... In 2014, Yeh (Department of Information Management, National Dong Hwa University) proposed multi-server scheme using Rivest–Shamir–Adleman (RSA). However, we discover this scheme cannot achieve two basic properties, mutual authentication and session-key agreement. In this paper, we concretely demonstrate that discovery and propose a different provable version using elliptic curve cryptosystem in multi-server environment which overcomes the limitation of Yeh’s scheme and satisfies security and efficiency.

Part 5: Virtual and Augmented Reality II; International audience; There are many techniques for locomotion and navigation that can support the exploration of large virtual environments in a limited physical area. Previous studies focused on measuring curvature gains and bending gains applied to the walking direction in the real world. However, the effects of different moving techniques and their relationship with shapes and patterns of virtually moving paths have not been studied extensively before. In this study, we present our experimental results on how users perceive two different traveling-in-place techniques with different bending gains of moving paths using a hybrid electric bike simulator. Moreover, the impact of different factors including road textures, road widths, and road curve directions and their relationships with the techniques are investigated. Generally, users could travel along a curve without noticing with a point of subjective equality (PSE) at bending angle $$\beta $$ = 1.42$$^\circ $$, and a just-noticeable difference (JND) of 0.75$$^\circ $$ for a movement at around 20 km/h during 5 s. In addition, movement technique, curve direction, and future travel path significantly affected how they perceived the curvature of their travel path.

City anomalies such as residential fires or urban floods can cause a lot of damage each year from loss of human lives to negative effects on the productivity of the city. One possible way to deal with these anomalies in a quick and effective manner is to use the information provided by the network of citizens that is likely present at the scene of any serious anomaly. However, this source of information may be of varying quality and there may be too much data for human operators to manually inspect in a speedy manner. This motivates us to propose an architecture to effectively make use of the network of citizens to deal with city anomalies. At the center of our architecture is a neural network that automatically classify incoming image data and use this information to assist anomaly handling efforts. In order for our classifier to work well, we have also collected a dataset of city anomalies specific to Vietnam, consisting of 9963 images. We experiment with several neural network models on our collected dataset. In overall, they all have decent accuracies with the best being MobileNet with 92.3% accuracy.

Searchable Symmetric Encryption (SSE) enables clients to securely store data on untrusted server while keeping the ability to search and update over the encrypted data efficiently. For practical purposes, we also need several additional properties to make our search function more versatile. In this paper, we introduce an extension type of SSE which we called Keyword-search Interval Dynamic SSE (KIDSSE). In KIDSSE we can search for encrypted files containing a keyword that occurs in a queried range. We also construct a solution for KIDSSE and show that our scheme maintains all important security properties such as forward privacy, backward privacy.

This paper presents an overview of the ImageCLEF 2019 lab, organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data, (ii) a lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (iii) a new security task addressing the problems of automatically identifying forged content and retrieve hidden information, and (iv) a new coral task about segmenting and labeling collections of coral images for 3D modeling. The strong participation, with 235 research groups registering, and 63 submitting over 359 runs, shows an important interest in this benchmark campaign.

Generalized nucleus segmentation techniques can contribute greatly to reducing the time to develop and validate visual biomarkers for new digital pathology datasets. We summarize the results of MoNuSeg 2018 Challenge whose objective was to develop generalizable nuclei segmentation techniques in digital pathology. The challenge was an official satellite event of the MICCAI 2018 conference in which 32 teams with more than 80 participants from geographically diverse institutes participated. Contestants were given a training set with 30 images from seven organs with annotations of 21,623 individual nuclei. A test dataset with 14 images taken from seven organs, including two organs that did not appear in the training set was released without annotations. Entries were evaluated based on average aggregated Jaccard index (AJI) on the test set to prioritize accurate instance segmentation as opposed to mere semantic segmentation. More than half the teams that completed the challenge outperformed a previous baseline. Among the trends observed that contributed to increased accuracy were the use of color normalization as well as heavy data augmentation. Additionally, fully convolutional networks inspired by variants of U-Net, FCN, and Mask-RCNN were popularly used, typically based on ResNet or VGG base architectures. Watershed segmentation on predicted semantic segmentation maps was a popular post-processing strategy. Several of the top techniques compared favorably to an individual human annotator and can be used with confidence for nuclear morphometrics.

This paper presents an overview of the foreseen ImageCLEF 2019 lab that will be organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of ImageCLEF will run four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with newer data, (iii) a new Coral task about segmenting and labeling collections of coral images for 3D modeling, and (iv) a new Security task addressing the problems of automatically identifying forged content and retrieve hidden information. The strong participation, with over 100 research groups registering and 31 submitting results for the tasks in 2018 shows an important interest in this benchmarking campaign and we expect the new tasks to attract at least as many researchers for 2019. acceptedVersion

With nonstop development of communication technologies, all aspects of social life continuously change and so do network systems. When establishing connection is easy, the convenience of online-service receives many users’ attentions, for example, the patients directly access medical system to be advised by doctors at any time. Therefore, user authentication scheme is necessary when we want to provide privacy and security for working sessions. Storing a password list for verification is an old method and not secure. This list can be easily leaked, and adversary can launch an offline password-guessing attack. In addition, information exchanged between user and server needs being prevented from attacker’s decryption. It can be said that current authentication schemes are unsuitable for new security standard. We need a strong user authentication scheme using new approach to overcome existing limitations and guarantee time efficiency. In this paper, we make a design with Chebyshev polynomial to achieve our goals and resist some kinds of attacks.

This paper describes the work of DCU research team in collaboration with University of Science, Vietnam, and University of Bergen, Norway at the Lifelog task of NTCIR-14. In this paper, a new interactive retrieval engine is described that supports faceted retrieval and we present the results of an initial experiment with four users. Following this initial experiment, we implement a list of changes for a revised interactive retrieval engine for the LSC2019 comparative evaluation competition. The interactive retrieval system we describe utilises the wide range of lifelog metadata provided by the task organisers to develop an extensive faceted retrieval system.

Many categories of the illusion of self-motion have been widely studied with the potential support of virtual reality. However, the effects of directly and indirectly powering the movement on the possibility of perceiving changes in moving speed and their relationship with sensory feedback on users' speed change perception have not been investigated before. In this paper, we present the results of our user study on the difference in perceiving changes in moving speed between two different movement techniques: "pedaling" and "throttling". We also explore the effects of different velocity gains, accelerations and speeds of airflow, and their interactions with the movement techniques on users' perception of speed changes in addition to user performance and perception. We built a bike simulator that supports both of the movement techniques and provides sensory feedback. In general, "pedaling" gave users more possibility to perceive changes in moving velocity than "throttling".

Lifelogging data provides useful insight understanding about our lives during daily activities. Thus, it is essential to develop a system to assist users to retrieve events or memories from lifelogging data from ad-hoc text queries. In this paper, we first propose a method to process lifelogging data by grouping images into visual shots and clusters, then extract semantic concepts on scene category and attributes, entities, and actions. We then develop a query system that supports 4 main types of query conditions: temporal, spatial, entity and action, and extra data criteria. Our system is expected to efficiently assist users to search for past moments in daily logs.

In this paper, we propose our method for vehicle detection with multiple adaptive vehicle detectors and velocity estimation with landmark-based scanlines. Inspired by the idea for tiny object detection, we use Faster R-CNN with Resnet-101 to create different specialized vehicle detectors corresponding to different levels of details and poses. We propose a heuristic to check the fitness of a particular vehicle detector to a specific region in camera's view by the mean velocity direction and the mean object size. By this way, we can determine an adaptive set of appropriate vehicle detectors for each region in camera's view. Thus our system is expected to detect vehicles with high accuracy, both in precision and recall, even with tiny objects. We exploit the U.S. road rules for the length and distance of broken white lines on roads to propose our method for vehicle's velocity estimation using such landmarks. We determine equally-distributed scanlines, virtual parallel lines that are nearly-perpendicular to the road direction, with reference to the line connecting the corresponding ends of multiple broken white lines. From the timespan for a vehicle to cross two consecutive virtual scanlines, we can calculate the average vehicle's velocity within that road segment. We also refine the speed estimation by detecting when a vehicle stops at a traffic light, and smooth the results with a moving average filter. Experiments on the dataset of Traffic Flow Analysis from NVIDIA AI City Challenge 2018 show that our method achieves the perfect detect rate of 100%, the average velocity difference of 6.9762 mph on freeways, and 8.9144 mph on both freeways and urban roads.

At DAVIS-2016 Challenge, many state-of-art video segmentation methods achieve potential results, but they still much depend on annotated frames to distinguish between background and foreground. It takes a lot of time and efforts to create these frames exactly. In this paper, we introduce a method to segment objects from video based on keywords given by user. First, we use a real-time object detection system - YOLOv2 to identify regions containing objects that have labels match with the given keywords in the first frame. Then, for each region identified from the previous step, we use Pyramid Scene Parsing Network to assign each pixel as foreground or background. These frames can be used as input frames for Object Flow algorithm to perform segmentation on entire video. We conduct experiments on a subset of DAVIS-2016 dataset in half the size of its original size, which shows that our method can handle many popular classes in PASCAL VOC 2012 dataset with acceptable accuracy, about 75.03%. We suggest widely testing by combining other methods to improve this result in the future.