Search

It is necessary to develop an efficient and secure mechanism to verify customers digitally for various online transactions. Integrating biometric solutions into the online user registration and verification processes is a promising trend for electronic Know Your Customer (eKYC) systems. However, Deepfake or face manipulation techniques may become a threat for eKYC with face authentication. In this paper, we introduce this potential attack of Deepfake on eKYC by swapping and manipulating faces between source and target faces. We then propose to augment the security for current eKYC systems with Deepfake detection. We conduct the experiments on the 10K video clips in the private test of Deepfake Detection Challenge 2020, and our method, following the Capsule-forensics approach, achieves the Logloss score of 0.5189, among the top 6% best results among the 2114 teams worldwide. This result demonstrates that our deepfake detection algorithm can be a promising method to provide extra protection for eKYC solutions with face registration and authentication.

In everyday life, authentication is an indispensable process of human activities. Bio-metric authentication system is one of the effective solutions, because it uses human-based features, instead of other traditional features, such as pin, password, etc. However, to apply a face authentication system in practical applications, we need to ensure that the system must not try to recognize the face of an unknown person into known categories, meaning we need to reject faces of unknown people in our application. In this paper, we present the limitations of recent Deep Learning based methods in Face Recognition tasks. We then propose two methods helping Face Recognition system have the ability to reject faces from unknown people by using Open-Set concepts. We conduct the experiments on a subset of CASIA-WebFace dataset, with a train set that includes 7000 images of 100 known people and a test set that includes both known and unknown people. Without rejecting unknown faces, the regular face recognition, i.e. the baseline method, yields the accuracy of only 45.9%, as the method tries to classify all face photos into known classes. Our proposed methods, which are combined deep network of Facenet system with recent Open Set methods, are called Learning Placeholder on Facenet (P-Facenet) and Facenet with OpenMax (O-Facenet). They achieve the accuracy of 83.6% and 88.5% respectively. This is a potential approach for authentication with face recognition to decrease the error rate of the model when recognizing faces of unknown people in the wild.

Polyps detection plays an important role in colonoscopy, cancer diagnosis, and early treatment. Many efforts have been made to improve the encoder-decoder framework using the global feature with an attention mechanism to enhance local features, helping to effectively segment diversity polyps. However, using only global information derived from the last encoder block leads to the loss of regional information from intermediate layers. Furthermore, defining the boundaries of some polyps is challenging because there is visual interference between the benign region and the polyps at the border. To address these problems, we propose two novel modules: the Cascading Context module (CCM) and the Attention Balance module (BAM), aiming to build an effective polyp segmentation model. Specifically, CCM combines the extracted regional information of the current layer and the lower layer, then pours it into the upper layer - fusing regional and global information analogous to a waterfall pattern. The BAM uses the prediction output of the adjacent lower layer as a guide map to implement the attention mechanism for the three regions separately: the background, polyp, and boundary curve. BAM enhances local context information when deriving features from the encoder block. Our proposed approach is evaluated on three benchmark datasets with six evaluation metrics for segmentation quality and gives competitive results compared to other advanced methods, for both accuracy and efficiency. Code is available at https://github.com/ntcongvn/CCBANet.

This paper presents an overview of the ImageCLEF 2020 lab that was organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2020. ImageCLEF is an ongoing evaluation initiative (first run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks, i.e., caption analysis, tuberculosis prediction, and medical visual question answering and question generation, (ii) a lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (iii) a coral task about segmenting and labeling collections of coral reef images, and (iv) a new Internet task addressing the problems of identifying hand-drawn user interface components. Despite the current pandemic situation, the benchmark campaign received a strong participation with over 40 groups submitting more than 295 runs.

Developing infrastructure and intelligent utilities for smart cities is an important trend in the world as well as in Vietnam. Thus, it is important to assist developers in building services for open data and smart city utilities. This motivates our proposal to develop a flexible platform with useful components, which can be integrated to develop these solutions quickly, to listen and analyze data from different social media sources with the diversification of data types, to provide open data providers in smart cities. Our method focuses on the ability to flexibly integrate artificial intelligence applications into the system to be able to both analyze effectively social events and serve smart cities in creating open data providers. We do not develop a particular system, but we create a platform, including different components, which are easy to be extended and integrated to create specific applications. To evaluate our platform, we develop four systems, including a face recognition system for celebrity recognition in news videos, an object detection system for brand logo recognition, a video highlighting system for summarizing football matches, and a text analysis system serving for keyword occurrences and emotional text analysis for admissions of universities. In these systems, we have collected and analyzed nearly 1000 videos from CNN, CBSN, FIFATV channels on YouTube, thousands of posts from admission pages of universities on Facebook. Each system gives a unique meaning to each specific situation for open data providers in smart cities.

This paper presents an overview of the 2020 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum---CLEF Labs 2020 in Thessaloniki, Greece. ImageCLEF is an ongoing evaluation initiative (run since 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF will organize four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data and adapted tasks, (iii) a Coral task about segmenting and labeling collections of coral images for 3D modeling, and a new (iv) Web user interface task addressing the problems of detecting and recognizing hand drawn website UIs (User Interfaces) for generating automatic code. The strong participation, with over 235 research groups registering and 63 submitting over 359 runs for the tasks in 2019 shows an important interest in this benchmarking campaign. We expect the new tasks to attract at least as many researchers for 2020.

The previous version of our retrieval system has shown some significant results in some retrieval tasks such as Lifelog’s moment retrieval tasks. In this paper, we adapt our platform to the Video Browser Showdown’s KIS and AVS tasks and present how our system performs in video search tasks. In addition to the smart features in our retrieval system that take advantage of the provided analysis data, we enhance the data with object color detection by employing Mask R-CNN and clustering. In this version of our search system, we try to extract the location information of the entities appearing in the videos and aim to exploit the spatial relationship between these entities. We also focus on designing efficient user interaction and a high-performance way to transfer data in the system to minimize the retrieval time.

Part 5: Virtual and Augmented Reality II; International audience; There are many techniques for locomotion and navigation that can support the exploration of large virtual environments in a limited physical area. Previous studies focused on measuring curvature gains and bending gains applied to the walking direction in the real world. However, the effects of different moving techniques and their relationship with shapes and patterns of virtually moving paths have not been studied extensively before. In this study, we present our experimental results on how users perceive two different traveling-in-place techniques with different bending gains of moving paths using a hybrid electric bike simulator. Moreover, the impact of different factors including road textures, road widths, and road curve directions and their relationships with the techniques are investigated. Generally, users could travel along a curve without noticing with a point of subjective equality (PSE) at bending angle $$\beta $$ = 1.42$$^\circ $$, and a just-noticeable difference (JND) of 0.75$$^\circ $$ for a movement at around 20 km/h during 5 s. In addition, movement technique, curve direction, and future travel path significantly affected how they perceived the curvature of their travel path.

Searchable Symmetric Encryption (SSE) enables clients to securely store data on untrusted server while keeping the ability to search and update over the encrypted data efficiently. For practical purposes, we also need several additional properties to make our search function more versatile. In this paper, we introduce an extension type of SSE which we called Keyword-search Interval Dynamic SSE (KIDSSE). In KIDSSE we can search for encrypted files containing a keyword that occurs in a queried range. We also construct a solution for KIDSSE and show that our scheme maintains all important security properties such as forward privacy, backward privacy.

This paper describes the work of DCU research team in collaboration with University of Science, Vietnam, and University of Bergen, Norway at the Lifelog task of NTCIR-14. In this paper, a new interactive retrieval engine is described that supports faceted retrieval and we present the results of an initial experiment with four users. Following this initial experiment, we implement a list of changes for a revised interactive retrieval engine for the LSC2019 comparative evaluation competition. The interactive retrieval system we describe utilises the wide range of lifelog metadata provided by the task organisers to develop an extensive faceted retrieval system.

Paraphrase identification is a core component for many significant tasks in natural language processing (e.g., text summarization, headline generation). A method suggested by Bach et al. for detecting Vietnamese paraphrase text using nine similarity metrics. The authors state that it is the first method for Vietnamese text. They evaluated the method on vnPara corpus with 3000 sentence pairs. However, this corpus is limited by collecting from few Vietnamese websites. Most other methods have focused on the English text. For instance, our previous method detected paraphrasing sentences by matching identical phrases and close words using Wordnet similarity. This method is unsuitable for Vietnamese due to the restriction of Wordnet corpora and morphological words in Vietnamese. Therefore, we extend the method to identify the paraphrase by proposing a SimVN metric which measures the similarity of two Vietnamese words. We evaluated the proposed method on the vnPara corpus. The result shows that the method achieves better accuracy (97.78%) comparing with the state-of-the-art method (accuracy = 89.10%). The proposed method then creates a high diversity paraphrase corpus with 3134 sentence pairs in eight main topics from the top fifteen popular Vietnamese news websites.

As inertial sensors are low-cost, easy-to-use, and can be integrated in wearable devices, they can be used to establish as a new modality for user authentication in the smart environment in which computing systems can recognize persons implicitly by their walking patterns. This motivates our proposal to use multi-region size Convolutional Neural Network to recognize users from their gait patterns recorded from accelerometers and gyroscopes in mobile and wearable devices.

Human Activity Recognition is one of the attractive topics to develop smart interactive environment in which computing systems can understand human activities in natural context. Besides traditional approaches with visual data, inertial sensors in wearable devices provide a promising approach for human activity recognition. In this paper, we propose novel methods to recognize human activities from raw data captured from inertial sensors using convolutional neural networks with either 2D or 3D filters. We also take advantage of hand-crafted features to combine with learned features from Convolution-Pooling blocks to further improve accuracy for activity recognition. Experiments on UCI Human Activity Recognition dataset with six different activities demonstrate that our method can achieve 96.95%, higher than existing methods.

Face clustering is a task to partition facial images into disjoint clusters. In this paper, we investigate a specific problem of face clustering in videos. Unlike traditional face clustering problem with a given collection of images from multiple sources, our task deals with set of face tracks with information about frame ID. Thus, we can exploit two kinds of prior knowledge about the temporal and spatial information from face tracks: sequence of faces in the same track and contemporary faces in the same frame. We utilize this forehand lore and characteristic of low rank representation to introduce a new light weight but effective method entitled Very Fast Sparse Clustering (VFSC). Since the superior speed of VFSC, the method can be adapted into large scale real-time applications. Experimental results with two public datasets (BF0502 and Notting-Hill), on which our proposed method significantly breaks the limits of not only speed but also accuracy clustering of state-of-the-art algorithms (up to 250 times faster and 10% higher in accuracy), reveal the imminent power of our approach.

In this paper, we propose two approaches to deal with the problems of video searching: ad-hoc video search and known item search. First, we propose to combine multiple semantic concepts extracted from multiple networks trained on many data domains. Second, to help user find exactly video shot that has been shown before, we propose a sketch based search system which detects and indexes many objects proposed by an object proposal algorithm. By this way, we not only leverage the concepts but also the spatial relations between them.

Advances in modern computing technology have enabled a wide variety of applications in many areas. Realizing that mobile devices such as smartphones and tablets are great learning platforms for educating young children, the authors create an interactive system that helps young kids learn new concepts via videos. By allowing kids to choose objects in the video that they are watching, the system provides relevant information about those objects. It has the effect of enhancing young kids’ understanding about new concepts in the video. The system employs Cross-based Local Multipoint Filtering (CLMF) for object selection and Support Vector Machine (SVM) for recognition. Experimental results show that our system has achieved 95 % recognition accuracy.

To get useful information on a landmark and to activate appropriate interaction related to that landmark can be a useful utility on mobile devices for travelers, especially in new visiting places. This motivates our proposal to use visual instance search to develop an interactive smart tourist guide system. Our aim is to provide not only a more accurate way to recommend a landmark and its information but also interesting and useful interactions around the landmark in order to seamlessly integrate real life interaction with the retrieved information. First, we develop our visual instance search framework that is optimized for speed and can achieve the accuracy approximating novel methods. Then, we apply our framework to the landmark recognition problem to replace the traditional approach of classification. Lastly, we apply our framework to our smart tourist guide system to identify a landmark, to provide its information as well as related interactions when given a landmark image. By incorporating visual instance search and interactive information, we can provide more accurate and seamlessly natural way of searching and interacting with landmarks for passengers and visitors in tourism.

The rapid growth of virtual reality in recent years has brought this technology to a wide variety of common users. Realizing the potential of this technology in building and planning cities; the authors introduce a system in which architectures are brought together into a 3-dimensional virtual environment in order to collaborate in building cities. This system uses the Oculus Rift as the VR device, combining with Leap Motion to detect user’s hand-gestures. The authors walkthrough all the details in building such systems including object modeling, communication protocols and gesture recognition technique.

Emotional and memorable moments are usually kept and shared on different online services such as Facebook, Flickr, Instagram, and Google Photos. As a result, one of users’ practical needs is to have their photos annotated automatically, especially with personalized tags. This motivates the authors to propose a system that can suggest personalized annotations for a photo uploaded to online services. Our system provides 2 major features. First, the system automatically recommends personalized annotations for newly uploaded photos based on visually similar photos uploaded in the past. Second, our system propagates manual annotations of users to other similar photos existed in their albums. To evaluate the performance of our system, we use the Oxford 5K Building Dataset and our own dataset consisting of personal photos collected from Facebook. Our systems achieves the mean Average Precision of 0.844 and 0.749 respectively on these two datasets. This demonstrates that our proposed solution can be potentially integrated as a useful utility or extension for online photo sharing services.

Deformable Part Models and Convolutional Neural Network are state-of-the-art approaches in object detection. While Deformable Part Models makes use of the general structure between parts and root models, Convolutional Neural Network uses all information of input to create meaningful features. These two types of characteristics are necessary for face detection. Inspired by this observation, first, we propose an extension of DPM by adaptively integrating CNN for face detection called DeepFace DPM and propose a new combined model for face representation. Second, a new way of calculating non-maximum suppression is also introduced to boost up detection accuracy. We use Face Detection Data Set and Benchmark to evaluate the merit of our method. Experimental results show that our method surpasses the highest result of existing methods for face detection on the standard dataset with 87.06i??% in true positive rate at 1000 number false positive images. Our method sheds a light in face detection which is commonly regarded as a saturated area.

Learn to read and understand a music sheet, then play it on a musical instrument are difficult tasks to most beginner music learners. This motivates the authors to propose Virtual Music Teacher, a system to assist beginner music learners in their learning process. By applying our proposed lightweight Optical Music Recognition algorithm to scan and recognize a music sheet, then combine with sound classifying technique, the proposed system can learn what note to be played next, then help a music learner to play it correctly. The experimental results on the dataset consisting of 15 musical scores for beginners show that the proposed system can classify with precision up to 99.9 % using multiple SVM classifiers approach, whereas the sound classifying technique using Fast Fourier Transform can classify note’s pitch recorded from a piano with precision up to 95.71 %. The system is implemented as an application on mobile devices and can be used to assist a music learner to play not only piano but other musical instruments as well.

In modern world, children need to get familiar with interactive toys to quickly improve their learning and imagination. Our approach is to add augmented information and interaction to common toys on the surface containing them, which is called Smart Playground. Popular methods use three color channels and local features to recognize objects. However, toys of children usually have various pictures with different colors drawn on many small components. Therefore depth data is useful in this case. Each toy usually have unique shape that is distinguishable from others. In this paper, we use an RGB-D sensor to collect information about both color and shape of objects. To learn the training set of toys, an approach of convolutional neural network is used to represent data (both color and depth) by high-level feature vectors. Using the combined results, the accuracy of 3D recognition is more than 90 %.

Face verification is a promising method for user authentication. Besides existing methods with deep convolutional neural networks to handle millions of people using powerful computing systems, the authors aim to propose an alternative approach of a lightweight scheme of convolutional neural networks (CNN) for face verification in realtime. Our goal is to propose a simple yet efficient method for face verification that can be deployed on regular commodity computers for individuals or small-to-medium organizations without super-computing strength. The proposed scheme targets unconstrained face verification, a typical scenario in reality. Experimental results on original data of Labeled Faces in the Wild dataset show that our best CNN found through experiments with 10 hidden layers achieves the accuracy of \((82.58 \pm 1.30)\,\%\) while many other instances in the same scheme can also approximate this result. The current implementation of our method can run at 60 fps and 235 fps on a regular computer with CPU-only and GPU configurations respectively. This is suitable for deployment in various applications without special requirements of hardware devices.

As of today, millions of people share messages via online social networks, some of which probably contain sensitive information. An adversary can collect these freely available messages and specifically analyze them for privacy leaks, such as the users’ location. Unlike other approaches that try to detect these leaks using complete message streams, we put forward a rule-based approach that works on single and very short messages to detect location leaks. We evaluated our approach based on 2817 tweets from the Tweets2011 data set. It scores significantly better (accuracy = 84.95 %) on detecting whenever a message reveals the user’s location than a baseline using machine learning and three extensions using heuristic. Advantages of our approach are not only to apply for online social network messages but also to extend for other areas (such as email, military, health) and for other languages.

The authors propose to develop a smart kiosk that plays the role of an identity selector activated implicitly when a user is approaching that kiosk. The identity of a user is recognized implicitly in background by a mobile/wearable device based on his or her gait features. Upon arriving at a smart kiosk, the authentication process is performed automatically with the current available user identity in his or her portable device. To realize our system, we propose a new secure authentication scheme compatible with gait-based continuous authentication that can resist against known attacks, including three-factor attacks. Furthermore, we also propose a method to recognize users from their moving patterns using multiple SVM classifiers. Experiments with a dataset with 38 people show that this method can achieve the accuracy up to 92.028i¾?%.

Perfect forward secrecy is considered as the most important standard to evaluate a strong authentication scheme. There are many results researched to achieve this property without using hard problems. Recently, the result of Chang et al has some advances such as, the correctness of schemes mutual authentication and session key agreement demonstrated in BAN-logic or the overheads reduction of system implementation. However, in this paper, we prove that their scheme is still vulnerable to impersonation attacks and session key leakage. To overcome those limitations and be practical, we use different notion to propose time efficient scheme conducted in experiment. Our proposed method can be applied for remote user authentication in various scenarios, including systems with user authentication using mobile or wearable devices.

Comics is rapidly developing and attracting a lot of people around the world. The problem is how a reader can find a translated version of a comics in his or her favorite language when he or she sees a certain comics page in another language. Therefore, in this paper, we propose a comics instance search based on Bag of Visual Words so that readers can find in a collection of translated versions of various comics with a single instance as a comics page in an arbitrary language. Our method is based on visual information and does not rely on textual information of comics. Our proposed system uses Apache Lucene to handle inverted index process to find comics pages with visual words and spatial verification using RANSAC to eliminate bad results. Experimental results on our dataset with 20 comics containing more than 270,000 images achieve the accuracy up to 77.5i¾?%. This system can be improved for building a commercial system that allows a reader easily search a multi-language collection of comics with a comics page as an input query.

Graphical user interfaces are usually first sketched out manually as hand drawing pictures and then must be realized by software developers to become prototypes or usable user interfaces. This motivates our proposal of a smart CASE tool that can understand hand drawing sketches of graphical user interfaces, including forms and their navigations, then automatically transform such draft designs into real user interfaces of a prototype or an application. By using the ideas of modeling and model-transformation in model driven engineering, the authors also propose a mechanism to generate graphical user interfaces as forms targeting different platforms. Experimental results show that our sketch recognition to understand hand drawing graphical user interfaces can achieve the accuracy of 97.86% and 95% in recognizing 7 common UI controls and arrows for navigation respectively. Our model transformation engine can generate user interfaces as forms for applications on 3 different platforms of mobile devices, including Windows Phone, Android, and iOS. This approach follows the trend to develop a new generation of smart CASE tools that can understand and interpret conceptual software design models into concrete software elements and components to assist the software development process in a natural way.

Derive from practical needs, especially in tourism industry; landmark recognition is an interesting and challenging problem on mobile devices. To obtain the robustness, landmarks are described by local features with many levels of invariance among which rotation invariance is commonly considered an important property. We propose to eliminate orientation normalization for local visual descriptors to enhance the accuracy in landmark recognition problem. Our experiments show that with three different widely used descriptors, including SIFT, SURF, and BRISK, our idea can improve the recognition accuracy from 2.3 to 12.6% while reduce the feature extraction time from 2.5 to 11.1%. This suggests a simple yet efficient method to boost the accuracy with different local descriptors with orientation normalization in landmark recognition applications.

The authors propose a system that supports children to learn new concepts of familiar topics via their sketches on an interaction surface. The proposed system has two main subcomponents: a system of interaction surface with touch detection from depth images captured by a Kinect and a sketch recognition module based on the idea of bag-of-word model. The system provides a natural and intuitive interface for children because they can learn new concepts via sketching. With the dataset of 70 common concepts, the accuracy of the sketch recognition is 78.21% and the average response time to recognize a sketch is 0.86s. The sketch database can also be easily customized to teach new concepts to children.

Online transactions with mobile devices through internet environment have become popular worldwide. Therefore, many authentication schemes have been proposed to protect users from various potential attacks in e-transactions with online service providers from mobile devices. In 2013, Khan et. al. propose a key-hash based scheme on mobile devices to resist known kinds of attacks that previous schemes cannot resist. However, we prove that Khan et. al.’s scheme still cannot withstand impersonation, denial of service, and three-factor attacks. This motivates our proposal of an improved scheme to further overcome the found limitations in Khan’s scheme. The main idea of our proposed method is that the user ID and the secret key of the server are hashed together to prevent user impersonation. We also prove that our method can also resist against known attacks, such as server and user impersonation attack, replay attack, password guessing attack, malicious user attack, mobile device loss attack, attacks due to ID theft, attacks using login request.

Mobile devices (e.g., PDA, mobile phone, tablet, and notebook PC) become necessary for a convenient and modern life. So, we can use them to access services, for examples online shopping, internet banking. In such insecure environment, we see that communications are more and more essential because they defend users and providers against illegitimate adversaries. Recently, Shin et al have proposed scientific paper entitled ’A Remote User Authentication Scheme with Anonymity for Mobile Devices’ to enhance security for remote user authentication. They claimed that their scheme is truly more secure than previous ones and it can resist various attacks. However, it is not true because their scheme’s vulnerable to insider, impersonation and replay attacks. In this paper, we present an improvement to their scheme to isolate such problems.