Search

This paper proposes a novel algorithm of discovering the structure of a kaleidoscopic imaging system that consists of multiple planar mirrors and a camera. The kaleidoscopic imaging system can be recognized as the virtual multi-camera system and has strong advantages in that the virtual cameras are strictly synchronized and have the same intrinsic parameters. In this paper, we focus on the extrinsic calibration of the virtual multi-camera system. The problems to be solved in this paper are two-fold. The first problem is to identify to which mirror chamber each of the 2D projections of mirrored 3D points belongs. The second problem is to estimate all mirror parameters, i.e., normals, and distances of the mirrors. The key contribution of this paper is to propose novel algorithms for these problems using a single 3D point of unknown geometry by utilizing a kaleidoscopic projection constraint, which is an epipolar constraint on mirror reflections. We demonstrate the performance of the proposed algorithm of chamber assignment and estimation of mirror parameters with qualitative and quantitative evaluations using synthesized and real data. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of reconstructing 3D poses of multiple people from a few calibrated camera views. The main challenge of this problem is to find the cross-view correspondences among noisy and incomplete 2D pose predictions. Most previous methods address this challenge by directly reasoning in 3D using a pictorial structure model, which is inefficient due to the huge state space. We propose a fast and robust approach to solve this problem. Our key idea is to use a multi-way matching algorithm to cluster the detected 2D poses in all views. Each resulting cluster encodes 2D poses of the same person across different views and consistent correspondences across the keypoints, from which the 3D pose of each person can be effectively inferred. The proposed convex optimization based multi-way matching algorithm is efficient and robust against missing and false detections, without knowing the number of people in the scene. Moreover, we propose to combine geometric and appearance cues for cross-view matching. Finally, an efficient tracking method is proposed to track the detected 3D poses across the multi-view video. The proposed approach achieves the state-of-the-art performance on the Campus and Shelf datasets, while being efficient for real-time applications. [ABSTRACT FROM AUTHOR]

Estimating the pose of a calibrated camera relative to a 3D point set from one image is an important task in computer vision. Perspective-n-Point algorithms are often used if perfect 2D-3D correspondences are known. However, it is difficult to determine 2D-3D correspondences perfectly, and then the simultaneous pose and correspondence determination problem is needed to be solved. Early methods aimed to solve this problem by local optimization. Recently, several new methods are proposed to globally solve this problem by using branch-and-bound (BnB) method, but they tend to be slow because the time complexity of the BnB-based methods is exponential to the dimensionality of the parameter space, and they directly search the 6D parameter space. In this paper, we propose to decompose the joint searching into two separate searching processes by introducing a rotation invariant feature (RIF). Specifically, we construct RIFs from the original 3D and 2D point sets and search for the globally optimal translation to match these two RIFs first. Then, the original 3D point set is translated and matched with the 2D point set to find a globally optimal rotation. Experiments on challenging data show that the proposed method outperforms state-of-the-art methods in terms of both speed and accuracy. [ABSTRACT FROM AUTHOR]

Due to the lack of distinctiveness to the background, robust monocular 6DOF pose tracking of less-distinct objects remains an important open problem. In this paper, we firstly analyze the object distinctiveness in the tracking process. Then, we propose a novel contour-part model based robust monocular pose tracking method for less-distinct objects. This paper uses the traditional contour feature for pose tracking in a novel strategy called contour part model. First, the contour part model is built by segmenting the projected contour rendered from the 3D model into contour segments of a certain length adaptively according to the Shi-Tomasi cornerness scores. Then, the correspondence is detected in the input image for each contour part by gradient orientation based template matching. Finally, pose tracking is achieved by solving the PnP problem. Experiment results on semi-synthetic and real images show that the proposed method performs better than the existing methods when pose tracking less-distinct objects and shows great robustness toward interference. Additionally, we combine the edge feature and the regional feature in a simple strategy. The overall performance of the fused method is boosted according to the experimental results on the RBOT dataset. The fact shows that the fusion of the edge and regional features has great potential for improving tracking performance. [ABSTRACT FROM AUTHOR]

With the development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary value of AR is to promote the fusion of digital contents and real-world environments, however, studies on how this fusion will influence the Quality of Experience (QoE) of these two components are lacking. To achieve better QoE of AR, whose two layers are influenced by each other, it is important to evaluate its perceptual quality first. In this paper, we consider AR technology as the superimposition of virtual scenes and real scenes, and introduce visual confusion as its basic theory. A more general problem is first proposed, which is evaluating the perceptual quality of superimposed images, i.e., confusing image quality assessment. A ConFusing Image Quality Assessment (CFIQA) database is established, which includes 600 reference images and 300 distorted images generated by mixing reference images in pairs. Then a subjective quality perception experiment is conducted towards attaining a better understanding of how humans perceive the confusing images. Based on the CFIQA database, several benchmark models and a specifically designed CFIQA model are proposed for solving this problem. Experimental results show that the proposed CFIQA model achieves state-of-the-art performance compared to other benchmark models. Moreover, an extended ARIQA study is further conducted based on the CFIQA study. We establish an ARIQA database to better simulate the real AR application scenarios, which contains 20 AR reference images, 20 background (BG) reference images, and 560 distorted images generated from AR and BG references, as well as the correspondingly collected subjective quality ratings. Three types of full-reference (FR) IQA benchmark variants are designed to study whether we should consider the visual confusion when designing corresponding IQA algorithms. An ARIQA metric is finally proposed for better evaluating the perceptual quality of AR images. Experimental results demonstrate the good generalization ability of the CFIQA model and the state-of-the-art performance of the ARIQA model. The databases, benchmark models, and proposed metrics are available at: https://github.com/DuanHuiyu/ARIQA. [ABSTRACT FROM AUTHOR]

Disguised tailing and visual monitoring of suspicious mobile targets is a promising application of security unmanned aerial vehicles (UAVs). But trajectory planning is non-trivial, especially for fixed-wing UAVs with more constrained maneuverability and dynamic models. This paper proposes a new framework to optimize collectively the propulsion power and the three-dimensional (3D) trajectory of a solar-powered, fixed-wing UAV on a disguised tailing and video surveillance mission. The multi-objective optimization strikes a balance between distance keeping, elevation variance, and power efficiency. A key aspect is that we develop a new propulsion power model of the fixed-wing UAV by analyzing the forces undergone while the UAV is ascending or descending. Another important aspect is a series of non-trivial reformulations, which convexify the multi-objective problem progressively with increasingly tightening linear approximation and solve the problem with a polynomial time-complexity. Our algorithm can control the trajectory of the UAV on-the-fly. Simulations confirm that the algorithm outperforms existing schemes in terms of visual disguise and power efficiency. The fixed-wing UAV also demonstrates its advantage of energy efficiency and sustainability to elongate the surveillance mission, over its rotary-wing counterpart. [ABSTRACT FROM AUTHOR]

The iterative closest point (ICP) algorithm and its variants are a fundamental technique for rigid registration between two point sets, with wide applications in different areas from robotics to 3D reconstruction. The main drawbacks for ICP are its slow convergence, as well as its sensitivity to outliers, missing data, and partial overlaps. Recent work such as Sparse ICP achieves robustness via sparsity optimization at the cost of computational speed. In this paper, we propose a new method for robust registration with fast convergence. First, we show that the classical point-to-point ICP can be treated as a majorization-minimization (MM) algorithm, and propose an Anderson acceleration approach to speed up its convergence. In addition, we introduce a robust error metric based on the Welsch’s function, which is minimized efficiently using the MM algorithm with Anderson acceleration. On challenging datasets with noises and partial overlaps, we achieve similar or better accuracy than Sparse ICP while being at least an order of magnitude faster. Finally, we extend the robust formulation to point-to-plane ICP, and solve the resulting problem using a similar Anderson-accelerated MM strategy. Our robust ICP methods improve the registration accuracy on benchmark datasets while being competitive in computational time. [ABSTRACT FROM AUTHOR]

This paper considers and solves the problem of estimating camera pose given a pair of point-tangent correspondences between a 3D scene and a projected image. The problem arises when considering curve geometry as the basis of forming correspondences, computation of structure and calibration, which in its simplest form is a point augmented with the curve tangent. We show that while the resectioning problem is solved with a minimum of three points given the intrinsic parameters, when points are augmented with tangent information only two points are required, leading to substantial robustness and computational savings, e.g., as a minimal engine within ransac. In addition, algorithms are developed to find a practical solution shown to effectively recover camera pose using synthetic and real datasets. This technology is intended as a building block of curve-based structure from motion systems, allowing new views to be incrementally registered to a core set of views for which relative pose has been computed. [ABSTRACT FROM AUTHOR]

Depth estimation from a camera is an important task for 3D perception. Recently, without using the labeled ground truth of depth map, a self-supervised deep learning network can use relative pose to synthesize the target image from the reference image, and the photometric error between synthesized reference image and real one is used as self-supervisory signal. In this paper, we propose a novel self-supervised depth estimation network, which takes advantage of the quadtree constraint to optimize the depth estimation network. Based on the quadtree constraint, the photometric loss and depth loss of quadtree are proposed. In order to solve the problem that multiple depth values in repeated structures and uniform texture regions can cause relatively low photometric loss, we use quadtree-based photometric loss, which calculates the averaged photometric loss in quadtree blocks instead of the pixel-wise loss. For the problem of imbalanced depth distribution, we use quadtree depth loss, which constrains the depth inconsistency within quadtree blocks. The depth estimation network is composed of deep fusion module and cross-layer feature fusion module, which can better extract the feature information of RGB image and sparse keypoints depths, and makes full use of the detail information of the shallow feature map and the semantic information of the deep feature map to enrich the feature information extraction. Experimental results demonstrate that our method outperforms the state-of-the-art approaches of depth estimation. [ABSTRACT FROM AUTHOR]

To realize a robust robotic grasping system for unknown objects in an unstructured environment, large amounts of grasp data and 3D model data for the object are required; the sizes of these data directly affect the rate of successful grasps. To reduce the time cost of data acquisition and labeling and increase the rate of successful grasps, we developed a self-supervised learning mechanism to control grasp tasks performed by manipulators. First, a manipulator automatically collects the point cloud for the objects from multiple perspectives to increase the efficiency of data acquisition. The complete point cloud for the objects is obtained using the hand-eye vision of the manipulator and the truncated signed distance function algorithm. Then, the point cloud data for the objects are used to generate a series of six-degrees-of-freedom grasp poses, and the force-closure decision algorithm is used to add the grasp quality label to each grasp pose to realize the automatic labeling of grasp data. Finally, the point cloud in the gripper closing area corresponding to each grasp pose is obtained and used to train the grasp-quality classification model for the manipulator. The results of performing actual grasping experiments demonstrate that the proposed self-supervised learning method can increase the rate of successful grasps for the manipulator. Note to Practitioners—Most of the existing grasp planning methods of the manipulator are based on public datasets or simulation data to train model algorithms. Owing to the limited types of objects, the limited amount of data in the public datasets, and the lack of real sensor noise in the simulation data, the robustness of the trained algorithm model is insufficient, and it is difficult to apply to unstructured production environments. To solve the above problems, we propose a 6-DOF capture planning method based on self-supervised learning and introduce a self-supervised learning mechanism to solve the problem of grasp data acquisition in real scenes. The manipulator automatically collects object data from multiple perspectives, performs desktop-level 3D reconstruction, and finally uses the force-closure decision algorithm to automatically label the data in order to realize automatic acquisition and labeling of the grasp data in a real scenario. Preliminary experiments show that this method can obtain high-quality grasp data and can be applied to grasp operations in real multi-target and cluttered environments. However, it has not been tested in actual production environments. This paper focuses on the data acquisition module in the 6-DOF grasp planning framework. In future research, we will design a more efficient grasp planning module to improve the grasp efficiency of the manipulator. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicle (UAV) communications and networks are of utmost concern. However, they have challenges such as the limited on-board energy and restricted transmit power. In this paper, we study a UAV-enabled communication scenario that a set of UAVs perform a virtual antenna array (VAA) to communicate with different remote base stations (BSs) by using collaborative beamforming (CB). To achieve a better transmission performance, the UAV elements can fly to optimal positions by using optimal speeds and adjust to optimal excitation current weights for performing CB transmissions. However, there are some trade-offs between energy consumption and transmission performance. Thus, we formulate a time and energy minimization communication multi-objective optimization problem (TEMCMOP) of CB in UAV networks to simultaneously minimize the total transmission time, total performing time of VAAs and total motion and hovering energy consumptions of UAVs by jointly optimizing the positions, flight speeds and excitation current weights of UAVs, as well as the order of communicating with different BSs. Due to the complexity and NP-hardness of the formulated TEMCMOP, we propose an improved multi-objective ant lion optimization (IMOALO) algorithm with chaos-opposition based learning solution initialization and hybrid solution update operators to solve the problem. Simulation results verify that the proposed IMOALO can effectively solve the formulated TEMCMOP and it has better performance than some other benchmark approaches. [ABSTRACT FROM AUTHOR]

We compare physical and virtual reality (VR) versions of simple data visualizations and explore how the addition of virtual annotation and filtering tools affects how viewers solve basic data analysis tasks. We report on two studies, inspired by previous examinations of data physicalizations. The first study examines differences in how viewers interact with physical hand-scale, virtual hand-scale, and virtual table-scale visualizations and the impact that the different forms had on viewer’s problem solving behavior. A second study examines how interactive annotation and filtering tools might support new modes of use that transcend the limitations of physical representations. Our results highlight challenges associated with virtual reality representations and hint at the potential of interactive annotation and filtering tools in VR visualizations. [ABSTRACT FROM AUTHOR]

High harmonic contents of the stator currents due to uncontrolled $x-y$ subspace dramatically overshadow the performance of the switching-table-based direct torque control (ST-DTC) strategy for multiphase drives. The concept of ST-DTC based on virtual voltage vectors (VVs) has been frequently developed to alleviate this problem. However, VVs with fixed duty ratios are incapable of compensating inherent machine/converter asymmetries and dead time harmonics, which are mapped into the $x-y$ subspace. To solve this problem, this article develops a dynamic duty-ratio-based DTC technique for six-phase induction machine drives, where the duty ratios of the selected VVs are not constant anymore. These duty ratios are updated based on the $x-y$ voltage commands arising from the closed-loop $x-y$ current controllers over every sampling period. The attained merits over and above the conventional fixed duty-ratio-based DTC schemes include effective $x-y$ current cancellation without increase in average switching frequency and decrease in dc-link utilization. Experimental results are presented to validate the effectiveness of the proposed control technique. [ABSTRACT FROM AUTHOR]

We propose a deep fine-grained multi-level fusion architecture for monocular 3D object detection, with an additionally designed anti-occlusion optimization process. Conventional monocular 3D object detection methods usually leverage geometry constraints such as keypoints, object shape relationships, and 3D to 2D optimizations to offset the lack of accurate depth information. However, these methods still struggle against directly extracting rich information for fusion from the depth estimation. To solve the problem, we integrate the monocular 3D features with the pseudo-LiDAR filter generation network between fine-grained multi-level layers. Our network utilizes the inherent multi-scale and promotes depth and semantic information flow in different stages. The new architecture can obtain features that incorporate more reliable depth information. At the same time, the problem of occlusion among objects is prevalent in natural scenes yet remains unsolved mainly. We propose a novel loss function that aims at alleviating the problem of occlusion. Extensive experiments have proved that the framework demonstrates a competitive performance, especially for the complex scenes with occlusion. [ABSTRACT FROM AUTHOR]

Interconnect delay is a key factor that affects the chip performance in layer assignment. Particularly in the advanced process technologies of 5 nm and beyond, interconnect delay has grown significantly due to the increase of circuit scale. Moreover, coupling effect existed in wires reduces the accuracy of delay evaluation. On the other hand, the size of vias is often ignored in layer assignment, which enlarges the mismatch between global routing and detailed routing. To solve these problems, we propose VPT, a timing-aware layer assignment algorithm considering via pillars, which includes the following five key techniques: 1) via pillar structure combined with nondefault-rule (NDR) wires is adopted to form a net delay optimization system for advanced process technologies; 2) a synthetical model that can adapt to varying types and sizes of both vias and wires is designed to evaluate overflow effectively; 3) a sorting strategy is devised to reduce uncertainty of layer assignment flow and improve stability of the proposed algorithm; 4) an awareness strategy based on multiaspect congestion assessment is designed to reduce overflow significantly; and 5) a net scalpel algorithm is devised to minimize the maximum delay of nets, so that the timing behaviors can be improved systematically. The experimental results on multiple benchmarks confirm that the proposed algorithm leads to lower delay and less overflow, while achieving the best solution quality among the existing algorithms with the shortest runtime. [ABSTRACT FROM AUTHOR]

Thanks to the feature of on-demand and flexible deployment, unmanned aerial vehicle base stations (UAV-BS) can enhance coverage and capacity beyond current 5 G cellular networks. The main challenge in deploying UAV-BS lies in the fact that the limited battery in UAB-BS cannot support long-time reliable services. Therefore, a serving UAV-BS being running out its power needs to be replaced with a new fully-charged UAV-BS. This is called UAV-BS replacement process. However, most existing UAV-BS replacement methods ignore the link outage problem during the replacement. Thus the outage-free replacement problem (OFRP) for UAV-BSs arises. We provide a method to calculate the outage probability of the UAV replacement approach. We further formulate the Outage Probability Minimization Problem (OPMP) for UAV-BS replacement. To solve this problem, a 3-Dimensional Outage-Free Replacement Approach (3D-OFRA) is presented, which exploits different heights of swapping UAV-BSs during the replacement process. More importantly, we provide the guidelines of determining the distance between two UAV-BSs to minimize the link outage possibility during the UAV-BS replacement process. Our numerical results show that with the recommended distance between two UAV-BSs based on 3D-OFRA, the outage probability of the served UEs approaches to zero, compared with 40% of the traditional direct replacement approach. [ABSTRACT FROM AUTHOR]

Human pose estimation and action recognition are related tasks since both problems are strongly dependent on the human body representation and analysis. Nonetheless, most recent methods in the literature handle the two problems separately. In this article, we propose a multi-task framework for jointly estimating 2D or 3D human poses from monocular color images and classifying human actions from video sequences. We show that a single architecture can be used to solve both problems in an efficient way and still achieves state-of-the-art or comparable results at each task while running with a throughput of more than 100 frames per second. The proposed method benefits from high parameters sharing between the two tasks by unifying still images and video clips processing in a single pipeline, allowing the model to be trained with data from different categories simultaneously and in a seamlessly way. Additionally, we provide important insights for end-to-end training the proposed multi-task model by decoupling key prediction parts, which consistently leads to better accuracy on both tasks. The reported results on four datasets (MPII, Human3.6M, Penn Action and NTU RGB+D) demonstrate the effectiveness of our method on the targeted tasks. Our source code and trained weights are publicly available at https://github.com/dluvizon/deephar. [ABSTRACT FROM AUTHOR]

Reliable markerless motion tracking of people participating in a complex group activity from multiple moving cameras is challenging due to frequent occlusions, strong viewpoint and appearance variations, and asynchronous video streams. To solve this problem, reliable association of the same person across distant viewpoints and temporal instances is essential. We present a self-supervised framework to adapt a generic person appearance descriptor to the unlabeled videos by exploiting motion tracking, mutual exclusion constraints, and multi-view geometry. The adapted discriminative descriptor is used in a tracking-by-clustering formulation. We validate the effectiveness of our descriptor learning on WILDTRACK T. Chavdarova et al., “WILDTRACK: A multi-camera HD dataset for dense unscripted pedestrian detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 5030–5039. and three new complex social scenes captured by multiple cameras with up to 60 people “in the wild”. We report significant improvement in association accuracy (up to 18 percent) and stable and coherent 3D human skeleton tracking (5 to 10 times) over the baseline. Using the reconstructed 3D skeletons, we cut the input videos into a multi-angle video where the image of a specified person is shown from the best visible front-facing camera. Our algorithm detects inter-human occlusion to determine the camera switching moment while still maintaining the flow of the action well. Website: http://www.cs.cmu.edu/~ILIM/projects/IM/Association4Tracking. [ABSTRACT FROM AUTHOR]