15 results on '"Frank P.-W. Lo"'
Search Results
2. Data-Driven Microscopic Pose and Depth Estimation for Optical Microrobot Manipulation
- Author
-
Benny Lo, Dandan Zhang, Guang-Zhong Yang, Frank P.-W. Lo, Jian-Qing Zheng, Wenjia Bai, and Engineering & Physical Science Research Council (EPSRC)
- Subjects
Technology ,Computer science ,Materials Science ,0205 Optical Physics ,Materials Science, Multidisciplinary ,Image processing ,pose estimation ,Physics, Applied ,Data-driven ,depth estimation ,Computer vision ,Nanoscience & Nanotechnology ,Electrical and Electronic Engineering ,0206 Quantum Physics ,Pose ,Monocular camera ,Science & Technology ,business.industry ,Physics ,deep learning ,Optics ,Atomic and Molecular Physics, and Optics ,image processing ,Electronic, Optical and Magnetic Materials ,0906 Electrical and Electronic Engineering ,Physics, Condensed Matter ,Physical Sciences ,Science & Technology - Other Topics ,Artificial intelligence ,optical microrobot ,business ,Biotechnology - Abstract
Optical microrobots have a wide range of applications in biomedical research for both in vitro and in vivo studies. In most microrobotic systems, the video captured by a monocular camera is the only way for visualizing the movements of microrobots, and only planar motion, in general, can be captured by a monocular camera system. Accurate depth estimation is essential for 3D reconstruction or autofocusing of microplatforms, while the pose and depth estimation are necessary to enhance the 3D perception of the microrobotic systems to enable dexterous micromanipulation and other tasks. In this paper, we propose a data-driven method for pose and depth estimation in an optically manipulated microrobotic system. Focus measurement is used to obtain features for Gaussian Process Regression (GPR), which enables precise depth estimation. For mobile microrobots with varying poses, a novel method is developed based on a deep residual neural network with the incorporation of prior domain knowledge about the optical microrobots encoded via GPR. The method can simultaneously track microrobots with complex shapes and estimate the pose and depth values of the optical microrobots. Cross-validation has been conducted to demonstrate the submicron accuracy of the proposed method and precise pose and depth perception for microrobots. We further demonstrate the generalizability of the method by adapting it to microrobots of different shapes using transfer learning with few-shot calibration. Intuitive visualization is provided to facilitate effective human-robot interaction during micromanipulation based on pose and depth estimation results. more...
- Published
- 2020
- Full Text
- View/download PDF
Catalog
3. EEG-based user identification system using 1D-convolutional long short-term memory neural networks
- Author
-
Yingnan Sun, Benny Lo, Frank P.-W. Lo, Engineering & Physical Science Research Council (E, and British Council (UK)
- Subjects
Technology ,0209 industrial biotechnology ,Biometrics ,User identification ,Property (programming) ,Computer science ,Speech recognition ,02 engineering and technology ,Electroencephalography ,Computer Science, Artificial Intelligence ,1D-Convolutional LSTM ,09 Engineering ,Identification system ,Long short term memory ,Engineering ,020901 industrial engineering & automation ,Artificial Intelligence ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,Artificial Intelligence & Image Processing ,Electroencephalograms (EEG) ,01 Mathematical Sciences ,Science & Technology ,Artificial neural network ,medicine.diagnostic_test ,Operations Research & Management Science ,General Engineering ,Engineering, Electrical & Electronic ,Computer Science Applications ,Identification (information) ,Computer Science ,020201 artificial intelligence & image processing ,08 Information and Computing Sciences - Abstract
Electroencephalographic (EEG) signals have been widely used in medical applications, yet the use of EEG signals as user identification systems for healthcare and Internet of Things (IoT) systems has only gained interests in the last few years. The advantages of EEG-based user identification systems lie in its dynamic property and uniqueness among different individuals. However, it is for this reason that manually designed features are not always adapted to the needs. Therefore, a novel approach based on 1D Convolutional Long Short-term Memory Neural Network (1D-Convolutional LSTM) for EEG-based user identification system is proposed in this paper. The performance of the proposed approach was validated with a public database consists of EEG data of 109 subjects. The experimental results showed that the proposed network has a very high averaged accuracy of 99.58%, when using only 16 channels of EEG signals, which outperforms the state-of-the-art EEG-based user identification methods. The combined use of CNNs and LSTMs in the proposed 1D-Convolutional LSTM can greatly improve the accuracy of user identification systems by utilizing the spatiotemporal features of the EEG signals with LSTM, and lowering cost of the systems by reducing the number of EEG electrodes used in the systems. more...
- Published
- 2019
- Full Text
- View/download PDF
4. Deep3DRanker: A Novel Framework for Learning to Rank 3D Models with Self-Attention in Robotic Vision
- Author
-
Frank P.-W. Lo, Jianing Qiu, Benny Lo, Yao Guo, and Yingnan Sun
- Subjects
Computer science ,business.industry ,Solid modeling ,Object (computer science) ,Machine learning ,computer.software_genre ,Domain (software engineering) ,Ranking (information retrieval) ,Encoding (memory) ,Graph (abstract data type) ,Augmented reality ,Learning to rank ,Artificial intelligence ,business ,computer - Abstract
Research on generating or processing point clouds has become an increasingly popular domain in robotic research due to its extensive applications, such as robotic grasping, augmented reality and autonomous vehicle navigation. In this paper, we explore a new research area on point clouds - Learning to rank 3D models captured from a single depth image. In the Learning To Rank (LTR) task, we aim at optimizing the order of a list of 3D models according to the given query. Inspired by the recent advances in Natural Language Processing (NLP), we propose a novel framework, namely Deep3DRanker, for ranking 3D models by leveraging graph-based encoding and self-attention mechanisms. Comprehensive experiments are conducted to validate our methods on publicly available YCB synthetic and YCB video datasets. The promising results have shown that our proposed framework is generic enough to be applicable with any combinations of randomly positioned, oriented, and unseen object items with accuracy ranging from 59.2% to 94.9%, which shows great potentials of the proposed framework for robotic applications, in particular, for making decisions under different circumstances. more...
- Published
- 2021
- Full Text
- View/download PDF
5. Egocentric Human Trajectory Forecasting with a Wearable Camera and Multi-Modal Fusion
- Author
-
Jianing Qiu, Lipeng Chen, Xiao Gu, Frank P.-W. Lo, Ya-Yen Tsai, Jiankai Sun, Jiaqi Liu, and Benny Lo
- Subjects
Human-Computer Interaction ,FOS: Computer and information sciences ,Control and Optimization ,Artificial Intelligence ,Control and Systems Engineering ,Mechanical Engineering ,Computer Vision and Pattern Recognition (cs.CV) ,Biomedical Engineering ,Computer Science - Computer Vision and Pattern Recognition ,Computer Vision and Pattern Recognition ,Computer Science Applications - Abstract
In this paper, we address the problem of forecasting the trajectory of an egocentric camera wearer (ego-person) in crowded spaces. The trajectory forecasting ability learned from the data of different camera wearers walking around in the real world can be transferred to assist visually impaired people in navigation, as well as to instill human navigation behaviours in mobile robots, enabling better human-robot interactions. To this end, a novel egocentric human trajectory forecasting dataset was constructed, containing real trajectories of people navigating in crowded spaces wearing a camera, as well as extracted rich contextual data. We extract and utilize three different modalities to forecast the trajectory of the camera wearer, i.e., his/her past trajectory, the past trajectories of nearby people, and the environment such as the scene semantics or the depth of the scene. A Transformer-based encoder-decoder neural network model, integrated with a novel cascaded cross-attention mechanism that fuses multiple modalities, has been designed to predict the future trajectory of the camera wearer. Extensive experiments have been conducted, with results showing that our model outperforms the state-of-the-art methods in egocentric human trajectory forecasting. more...
- Published
- 2021
- Full Text
- View/download PDF
6. Assessing individual dietary intake in food sharing scenarios with food and human pose detection
- Author
-
Jiabao Lei, Frank P.-W. Lo, Jianing Qiu, Benny Lo, Bill and Melinda Gates Foundation, Bill & Melinda Gates Foundation, and British Council (UK)
- Subjects
Food sharing ,Computer science ,business.industry ,Dietary intake ,020208 electrical & electronic engineering ,digestive, oral, and skin physiology ,02 engineering and technology ,Data science ,Individual based ,Work (electrical) ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Artificial Intelligence & Image Processing ,Artificial intelligence ,business - Abstract
Food sharing and communal eating are very common in some countries. To assess individual dietary intake in food sharing scenarios, this work proposes a vision-based approach to first capturing the food sharing scenario with a 360-degree camera, and then using a neural network to infer different eating states of each individual based on their body pose and relative positions to the dishes. The number of bites each individual has taken of each dish is then deduced by analyzing the inferred eating states. A new dataset with 14 panoramic food sharing videos was constructed to validate our approach. The results show that our approach is able to reliably predict different eating states as well as individual’s bite count with respect to each dish in food sharing scenarios. more...
- Published
- 2020
7. Image-based food classification and volume estimation for dietary assessment: a review
- Author
-
Benny Lo, Yingnan Sun, Frank P.-W. Lo, Jianing Qiu, Bill and Melinda Gates Foundation, and Bill & Melinda Gates Foundation
- Subjects
Dietary assessment ,Computer science ,Judgement ,Feature extraction ,030209 endocrinology & metabolism ,02 engineering and technology ,Field (computer science) ,03 medical and health sciences ,Deep Learning ,0302 clinical medicine ,Health Information Management ,Food classification ,Image Processing, Computer-Assisted ,0202 electrical engineering, electronic engineering, information engineering ,Humans ,Electrical and Electronic Engineering ,Nutritional epidemiology ,business.industry ,Deep learning ,Portion Size ,020206 networking & telecommunications ,Diet Records ,Diet ,Computer Science Applications ,Variety (cybernetics) ,Risk analysis (engineering) ,Food ,Artificial intelligence ,business ,Algorithms ,Biotechnology - Abstract
A daily dietary assessment method named 24-hour dietary recall has commonly been used in nutritional epidemiology studies to capture detailed information of the food eaten by the participants to help understand their dietary behaviour. However, in this self-reporting technique, the food types and the portion size reported highly depends on users’ subjective judgement which may lead to a biased and inaccurate dietary analysis result. As a result, a variety of visual-based dietary assessment approaches have been proposed recently. While these methods show promises in tackling issues in nutritional epidemiology studies, several challenges and forthcoming opportunities, as detailed in this study, still exist. This study provides an overview of computing algorithms, mathematical models and methodologies used in the field of image-based dietary assessment. It also provides a comprehensive comparison of the state of the art approaches in food recognition and volume/weight estimation in terms of their processing speed, model accuracy, efficiency and constraints. It will be followed by a discussion on deep learning method and its efficacy in dietary assessment. After a comprehensive exploration, we found that integrated dietary assessment systems combining with different approaches could be the potential solution to tackling the challenges in accurate dietary intake assessment. more...
- Published
- 2020
8. Point2Volume: A vision-based dietary assessment approach using view synthesis
- Author
-
Jianing Qiu, Frank P.-W. Lo, Benny Lo, and Yingnan Sun
- Subjects
Estimation ,Electrical & Electronic Engineering ,Dietary assessment ,Point (typography) ,Computer science ,Nutritional epidemiology ,business.industry ,Dietary intake ,020208 electrical & electronic engineering ,02 engineering and technology ,Machine learning ,computer.software_genre ,09 Engineering ,Computer Science Applications ,View synthesis ,Control and Systems Engineering ,10 Technology ,Structured interview ,0202 electrical engineering, electronic engineering, information engineering ,Artificial intelligence ,08 Information and Computing Sciences ,Electrical and Electronic Engineering ,business ,computer ,Information Systems - Abstract
Dietary assessment is an important tool for nutritional epidemiology studies. To assess the dietary intake, the common approach is to carry out 24-h dietary recall (24HR), a structured interview conducted by experienced dietitians. Due to the unconscious biases in such self-reporting methods, many research works have proposed the use of vision-based approaches to provide accurate and objective assessments. In this article, a novel vision-based method based on real-time three-dimensional (3-D) reconstruction and deep learning view synthesis is proposed to enable accurate portion size estimation of food items consumed. A point completion neural network is developed to complete partial point cloud of food items based on a single depth image or video captured from any convenient viewing position. Once 3-D models of food items are reconstructed, the food volume can be estimated through meshing. Compared to previous methods, our method has addressed several major challenges in vision-based dietary assessment, such as view occlusion and scale ambiguity, and it outperforms previous approaches in accurate portion size estimation. more...
- Published
- 2019
9. A deep learning approach on gender and age recognition using a single inertial sensor
- Author
-
Yingnan Sun, Frank P.-W. Lo, Benny Lo, Engineering & Physical Science Research Council (E, and British Council (UK)
- Subjects
gender recognition ,Technology ,Inertial frame of reference ,Biometrics ,Computer science ,Age recognition ,Feature extraction ,02 engineering and technology ,Machine learning ,computer.software_genre ,Cross-validation ,Engineering ,gait biometrics ,Inertial measurement unit ,Robustness (computer science) ,0202 electrical engineering, electronic engineering, information engineering ,Science & Technology ,business.industry ,Deep learning ,Soft biometrics ,020207 software engineering ,Engineering, Electrical & Electronic ,inertial sensors ,Computer Science ,soft biometrics ,020201 artificial intelligence & image processing ,Computer Science, Interdisciplinary Applications ,Artificial intelligence ,business ,computer - Abstract
Extracting human attributes, such as gender and age, from biometrics have received much attention in recent years. Gender and age recognition can provide crucial information for applications such as security, healthcare, and gaming. In this paper, a novel deep learning approach on gender and age recognition using a single inertial sensors is proposed. The proposed approach is tested using the largest available inertial sensor-based gait database with data collected from more than 700 subjects. To demonstrate the robustness and effectiveness of the proposed approach, 10 trials of inter-subject Monte-Carlo cross validation were conducted, and the results show that the proposed approach can achieve an averaged accuracy of 86.6%±2.4% for distinguishing two age groups: teen and adult, and recognizing gender with averaged accuracies of 88.6%±2.5 % and 73.9% ±2.8 % for adults and teens respectively. more...
- Published
- 2019
10. Assessing individual dietary intake in food sharing scenarios with a 360 camera and deep learning
- Author
-
Jianing Qiu, Frank P.-W. Lo, Benny Lo, Bill and Melinda Gates Foundation, and Bill & Melinda Gates Foundation
- Subjects
0301 basic medicine ,Technology ,Computer science ,Machine learning ,computer.software_genre ,Facial recognition system ,03 medical and health sciences ,0302 clinical medicine ,Engineering ,Segmentation ,030212 general & internal medicine ,Food sharing ,030109 nutrition & dietetics ,Science & Technology ,Artificial neural network ,business.industry ,Deep learning ,digestive, oral, and skin physiology ,Engineering, Electrical & Electronic ,object detection ,Image segmentation ,dietary intake assessment ,Object detection ,Computer Science ,360-degree video ,Computer Science, Interdisciplinary Applications ,Artificial intelligence ,business ,Transfer of learning ,computer - Abstract
A novel vision-based approach for estimating individual dietary intake in food sharing scenarios is proposed in this paper, which incorporates food detection, face recognition and hand tracking techniques. The method is validated using panoramic videos which capture subjects' eating episodes. The results demonstrate that the proposed approach is able to reliably estimate food intake of each individual as well as the food eating sequence. To identify the food items ingested by the subject, a transfer learning approach is designed. 4, 200 food images with segmentation masks, among which 1,500 are newly annotated, are used to fine-tune the deep neural network for the targeted food intake application. In addition, a method for associating detected hands with subjects is developed and the outcomes of face recognition are refined to enable the quantification of individual dietary intake in communal eating settings. more...
- Published
- 2019
11. Depth Estimation based on a Single Close-up Image with Volumetric Annotations in the Wild: A Pilot Study
- Author
-
Benny Lo, Frank P.-W. Lo, and Yingnan Sun
- Subjects
0301 basic medicine ,Estimation ,Technology ,Science & Technology ,Monocular ,Artificial neural network ,business.industry ,Computer science ,030106 microbiology ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Engineering, Electrical & Electronic ,Robotics ,Image segmentation ,Viewing angle ,Image (mathematics) ,Engineering, Mechanical ,03 medical and health sciences ,Engineering ,030104 developmental biology ,RGB color model ,Computer vision ,Artificial intelligence ,Focus (optics) ,business ,ComputingMethodologies_COMPUTERGRAPHICS - Abstract
A novel depth estimation technique based on a single close-up image is proposed in this paper for better understanding of the geometry of an unknown scene. Previous works focus mainly on depth estimation from global view information. Our technique, which is designed based on a deep neural network framework, utilizes monocular color images with volumetric annotations to train a two-stage neural network to estimate the depth information from close-up images. RGBVOL, a database of RGB images with volumetric annotations, has also been constructed by our group to validate the proposed methodology. Compared to previous depth estimation techniques, our method improves the accuracy of depth estimation under the condition that global cues of the scene are not available due to viewing angle and distance constraints. more...
- Published
- 2019
- Full Text
- View/download PDF
12. A Novel Vision-based Approach for Dietary Assessment using Deep Learning View Synthesis
- Author
-
Jianing Qiu, Frank P.-W. Lo, Yingnan Sun, Benny Lo, Bill and Melinda Gates Foundation, and Bill & Melinda Gates Foundation
- Subjects
Technology ,Computer science ,media_common.quotation_subject ,02 engineering and technology ,Machine learning ,computer.software_genre ,Engineering ,0202 electrical engineering, electronic engineering, information engineering ,media_common ,Science & Technology ,Artificial neural network ,Point (typography) ,business.industry ,Deep learning ,020208 electrical & electronic engineering ,3D reconstruction ,Engineering, Electrical & Electronic ,020206 networking & telecommunications ,Image segmentation ,Ambiguity ,View synthesis ,Dietary assessment ,Computer Science ,Structured interview ,Computer Science, Interdisciplinary Applications ,Artificial intelligence ,business ,computer - Abstract
Dietary assessment system has proven as an effective tool to evaluate the eating behavior of patients suffering from diabetes and obesity. To assess the dietary intake, the traditional method is to carry out a 24-hour dietary recall (24HR), a structured interview aimed at capturing information on food items and portion size consumed by participants. However, unconscious biases are developed easily due to individual's subjective perception in this self-reporting technique which may lead to inaccuracy. Thus, this paper proposed a novel vision-based approach for estimating the volume of food items based on deep learning view synthesis and depth sensing techniques. In this paper, a point completion network is applied to perform 3D reconstruction of food items using a single depth image captured from any convenient viewing angle. Compared to previous approaches, the proposed method has addressed several key challenges in vision-based dietary assessment, such as view occlusion and scale ambiguity. Experiments have been carried out to examine this approach and showed the feasibility of the algorithm in accurate estimation of food volume. more...
- Published
- 2019
- Full Text
- View/download PDF
13. Food volume estimation for quantifying dietary intake with a wearable camera
- Author
-
Frank P.-W. Lo, Benny Lo, Anqi Gao, Bill and Melinda Gates Foundation, and Bill & Melinda Gates Foundation
- Subjects
Convex hull ,Technology ,Computer science ,Wearable computer ,02 engineering and technology ,Iterative reconstruction ,Simultaneous localization and mapping ,01 natural sciences ,Engineering ,0202 electrical engineering, electronic engineering, information engineering ,Computer vision ,Polygon mesh ,Computer Science, Cybernetics ,Engineering, Biomedical ,Monocular ,Science & Technology ,business.industry ,020208 electrical & electronic engineering ,010401 analytical chemistry ,Engineering, Electrical & Electronic ,Object (computer science) ,0104 chemical sciences ,Computer Science ,Artificial intelligence ,business ,Volume (compression) - Abstract
A novel food volume measurement technique is proposed in this paper for accurate quantification of the daily dietary intake of the user. The technique is based on simul- taneous localisation and mapping (SLAM), a modified version of convex hull algorithm, and a 3D mesh object reconstruction technique. This paper explores the feasibility of applying SLAM techniques for continuous food volume measurement with a monocular wearable camera. A sparse map will be generated by SLAM after capturing the images of the food item with the camera and the multiple convex hull algorithm is applied to form a 3D mesh object. The volume of the target object can then be computed based on the mesh object. Compared to previous volume measurement techniques, the proposed method can measure the food volume continuously with no prior information such as pre-defined food shape model. Experiments have been carried out to evaluate this new technique and showed the feasibility and accuracy of the proposed algorithm in measuring food volume. more...
- Published
- 2017
14. Continuous systolic and diastolic blood pressure estimation utilizing long short-term memory network
- Author
-
Frank P.-W. Lo, Jiankun Wang, Charles Li, Jiyu Cheng, and Max Q.-H. Meng
- Subjects
Estimation ,Ground truth ,Ambulatory blood pressure ,Mean squared error ,Artificial neural network ,business.industry ,05 social sciences ,Blood Pressure ,Blood Pressure Determination ,Pattern recognition ,02 engineering and technology ,Long short term memory ,Memory, Short-Term ,Recurrent neural network ,Blood pressure ,0502 economics and business ,0202 electrical engineering, electronic engineering, information engineering ,Medicine ,050211 marketing ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,Biomedical engineering - Abstract
A novel blood pressure estimation method based on long short-term memory neural network, one of the recurrent neural networks being commonly used nowadays, is proposed in this paper for better chronic diseases monitoring. Along with the neural network, a newly proposed ambulatory blood pressure (ABP) processing technique called Two-stage Zero-order Holding (TZH) algorithm has also been presented in the paper. The proposed methodology has the advantages over traditional blood pressure estimation algorithms which are based on Pulse Transit time (PTT). The paper addresses the effectiveness of the algorithm by computing the Root-Mean-Squared Errors (RMSE) between the BP estimated and the ground truth. Our algorithm shows precise systolic blood pressure and diastolic blood pressure estimation with the average RMSE values in 2.751 mmHg and 1.604 mmHg respectively across the sample used. Experimental results suggest that BP estimation based on LSTM has great potential to be embedded into monitoring system for better accuracy and generalization. more...
- Published
- 2017
- Full Text
- View/download PDF
15. Double sensor complementary placement method to reduce motion artifacts in PPG using fast independent component analysis
- Author
-
Max Q.-H. Meng and Frank P.-W. Lo
- Subjects
Signal processing ,Computer science ,business.industry ,020208 electrical & electronic engineering ,0206 medical engineering ,Signal Processing, Computer-Assisted ,02 engineering and technology ,020601 biomedical engineering ,Signal ,Independent component analysis ,Reduction (complexity) ,Motion ,0202 electrical engineering, electronic engineering, information engineering ,Source separation ,Electronic engineering ,FastICA ,Humans ,Computer vision ,Artificial intelligence ,Artifacts ,Photoplethysmography ,business ,Algorithms - Abstract
A novel sensor placement method for better removal of motion artifacts (MA) from photoplethysmography (PPG) signal using Fast Independent Component Analysis (ICA) is proposed in this paper. The method enhances the determination of pulse transit time (PTT) of PPG signals. The design makes use of double reflectance mode based PPG probes, which are placed complementary to each other and on the two sides of a single finger. Furthermore, a novel indicator denoted as Separating Factor is proposed as well. It helps evaluating the performance of ICA with different sensors configuration. This paper then addresses the effectiveness of FastICA in motion artifacts reduction by using the novel method and normal sensors placement method to capture PPG. Results indicate that better independent source separation can be achieved and morphology of PPG signal is perfectly restored when using the method proposed in this paper. more...
- Published
- 2016
- Full Text
- View/download PDF
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.