Search

Pedestrian tracking is a significant problem in autonomous driving. The majority of studies carries out tracking in the image domain, which is not sufficient for many realistic applications like path planning, collision avoidance, and autonomous navigation. In this study, we address pedestrian tracking using stereo images and tracking-by-detection. Our framework comes in three primary phases: (1) people are detected in image space by the mask R-CNN detector and their positions in 3D-space are computed using stereo information; (2) corresponding detections are assigned to each other across consecutive frames based on visual characteristics and 3D geometry; and (3) the current positions of pedestrians are corrected using their previous states using an extended Kalman filter. We use our tracking-to-confirm-detection method, in which detections are treated differently depending on their confidence metrics. To obtain a high recall value while keeping a low number of false positives. While existing methods consider all target trajectories have equal accuracy, we estimate a confidence value for each trajectory at every epoch. Thus, depending on their confidence values, the targets can have different contributions to the whole tracking system. The performance of our approach is evaluated using the Kitti benchmark dataset. It shows promising results comparable to those of other state-of-the-art methods.

In this paper we present a novel approach for image orientation by combining relative rotations and tie points. First, we choose an initial image pair with enough correspondences and large triangulation angle, and we then iteratively add clusters of new images. The rotation of these newly added images is estimated from relative rotations by single rotation averaging. In the next step, a linear equation system is set up for each new image to solve the translation parameters with triangulated tie points which can be viewed in that new image, followed by a resection for refinement. Finally, we optimize the cluster of reconstructed images by local bundle adjustment. We show results of our approach on different benchmark datasets. Furthermore, we orient several larger datasets incl. unordered image datasets to demonstrate the robustness and performance of our approach.

Self-driving cars and robots that run autonomously over long periods of time need high-precision and up-to-date models of the changing environment. The main challenge for creating long term maps of dynamic environments is to identify changes and adapt the map continuously. Changes can occur abruptly, gradually, or even periodically. In this work, we investigate how dense mapping data of several epochs can be used to identify the temporal behavior of the environment. This approach anticipates possible future scenarios where a large fleet of vehicles is equipped with sensors which continuously capture the environment. This data is then being sent to a cloud based infrastructure, which aligns all datasets geometrically and subsequently runs scene analysis on it, among these being the analysis for temporal changes of the environment. Our experiments are based on a LiDAR mobile mapping dataset which consists of 150 scan strips (a total of about 1 billion points), which were obtained in multiple epochs. Parts of the scene are covered by up to 28 scan strips. The time difference between the first and last epoch is about one year. In order to process the data, the scan strips are aligned using an overall bundle adjustment, which estimates the surface (about one billion surface element unknowns) as well as 270,000 unknowns for the adjustment of the exterior orientation parameters. After this, the surface misalignment is usually below one centimeter. In the next step, we perform a segmentation of the point clouds using a region growing algorithm. The segmented objects and the aligned data are then used to compute an occupancy grid which is filled by tracing each individual LiDAR ray from the scan head to every point of a segment. As a result, we can assess the behavior of each segment in the scene and remove voxels from temporal objects from the global occupancy grid.

A novel calibration process of RICOH-THETA, full-view fisheye camera, is proposed which has numerous applications as a low cost sensor in different disciplines such as photogrammetry, robotic and machine vision and so on. Ricoh Company developed this camera in 2014 that consists of two lenses and is able to capture the whole surrounding environment in one shot. In this research, each lens is calibrated separately and interior/relative orientation parameters (IOPs and ROPs) of the camera are determined on the basis of designed calibration network on the central and side images captured by the aforementioned lenses. Accordingly, designed calibration network is considered as a free distortion grid and applied to the measured control points in the image space as correction terms by means of bilinear interpolation. By performing corresponding corrections, image coordinates are transformed to the unit sphere as an intermediate space between object space and image space in the form of spherical coordinates. Afterwards, IOPs and EOPs of each lens are determined separately through statistical bundle adjustment procedure based on collinearity condition equations. Subsequently, ROPs of two lenses is computed from both EOPs. Our experiments show that by applying 3*3 free distortion grid, image measurements residuals diminish from 1.5 to 0.25 degrees on aforementioned unit sphere.

We propose a new approach for the automatic detection of network structures in raster data. The model for the network structure is represented by a graph whose nodes and edges correspond to junction-points and to connecting line segments, respectively; nodes and edges are further described by certain parameters. We embed this model in the probabilistic framework of marked point processes and determine the most probable configuration of objects by stochastic sampling. That is, different graph configurations are constructed randomly by modifying the graph entity parameters, by adding and removing nodes and edges to/ from the current graph configuration. Each configuration is then evaluated based on the probabilities of the changes and an energy function describing the conformity with a predefined model. By using the Reversible Jump Markov Chain Monte Carlo sampler, a global optimum of the energy function is determined. We apply our method to the detection of river and tidal channel networks in digital terrain models. In comparison to our previous work, we introduce constraints concerning the flow direction of water into the energy function. Our goal is to analyse the influence of different parameter settings on the results of network detection in both, synthetic and real data. Our results show the general potential of our method for the detection of river networks in different types of terrain.

This paper proposes a novel approach for linear feature detection. The contribution is twofold: a novel model for spatial point processes and a new method for linear feature detection. It describes a linear feature as a string of points, represents all features in an image as a configuration of a spatial point process, and formulates feature detection as finding the optimal configuration of a spatial point process. Further, a prior term is proposed to favor straight linear configurations, and a data term is constructed to superpose the points on linear features. The proposed approach extracts straight linear features in a global framework. The paper reports ongoing work. As demonstrated in preliminary experiments, globally optimal linear features can be detected. National Natural Science Foundation of China/41071263 National Natural Science Foundation of China/41571335 Zhejiang Provincial Natural Science Foundation of China/LY13D010003 Key Laboratory for National Geographic Census and Monitoring National Administration of Surveying, Mapping and Geoinformation/2014NGCM

We propose a novel hierarchical approach for the classification of airborne 3D lidar points. Spatial and semantic context is incorporated via a two-layer Conditional Random Field (CRF). The first layer operates on a point level and utilises higher order cliques. Segments are generated from the labelling obtained in this way. They are the entities of the second layer, which incorporates larger scale context. The classification result of the segments is introduced as an energy term for the next iteration of the point-based layer. This framework iterates and mutually propagates context to improve the classification results. Potentially wrong decisions can be revised at later stages. The output is a labelled point cloud as well as segments roughly corresponding to object instances. Moreover, we present two new contextual features for the segment classification: the distance and the orientation of a segment with respect to the closest road. It is shown that the classification benefits from these features. In our experiments the hierarchical framework improve the overall accuracies by 2.3% on a point-based level and by 3.0% on a segment-based level, respectively, compared to a purely point-based classification.

Online multi-person tracking in image sequences is commonly guided by recursive filters, whose predictive models define the expected positions of future states. When a predictive model deviates too much from the true motion of a pedestrian, which is often the case in crowded scenes due to unpredicted accelerations, the data association is prone to fail. In this paper we propose a novel predictive model on the basis of Gaussian Process Regression. The model takes into account the motion of every tracked pedestrian in the scene and the prediction is executed with respect to the velocities of all interrelated persons. As shown by the experiments, the model is capable of yielding more plausible predictions even in the presence of mutual occlusions or missing measurements. The approach is evaluated on a publicly available benchmark and outperforms other state-of-the-art trackers.

In this paper we describe learning of a descriptor based on the Siamese Convolutional Neural Network (CNN) architecture and evaluate our results on a standard patch comparison dataset. The descriptor learning architecture is composed of an input module, a Siamese CNN descriptor module and a cost computation module that is based on the L2 Norm. The cost function we use pulls the descriptors of matching patches close to each other in feature space while pushing the descriptors for non-matching pairs away from each other. Compared to related work, we optimize the training parameters by combining a moving average strategy for gradients and Nesterov's Accelerated Gradient. Experiments show that our learned descriptor reaches a good performance and achieves state-of-art results in terms of the false positive rate at a 95 % recall rate on standard benchmark datasets.

driving. While much progress has been made in recent years, imaging conditions in natural outdoor environments are still very challenging for current reconstruction and recognition methods. In this paper, we propose a novel unified approach which reasons jointly about 3D scene flow as well as the pose, shape and motion of vehicles in the scene. Towards this goal, we incorporate a deformable CAD model into a slanted-plane conditional random field for scene flow estimation and enforce shape consistency between the rendered 3D models and the parameters of all superpixels in the image. The association of superpixels to objects is established by an index variable which implicitly enables model selection. We evaluate our approach on the challenging KITTI scene flow dataset in terms of object and scene flow estimation. Our results provide a prove of concept and demonstrate the usefulness of our method.

Complex activity modeling and identification of anomaly is one of the most interesting and desired capabilities for automated video behavior analysis. A number of different approaches have been proposed in the past to tackle this problem. There are two main challenges for activity modeling and anomaly detection: 1) most existing approaches require sufficient data and supervision for learning; 2) the most interesting abnormal activities arise rarely and are ambiguous among typical activities, i.e. hard to be precisely defined. In this paper, we propose a novel approach to model complex activities and detect anomalies by using non-parametric Gaussian Process (GP) models in a crowded and complicated traffic scene. In comparison with parametric models such as HMM, GP models are nonparametric and have their advantages. Our GP models exploit implicit spatial-temporal dependence among local activity patterns. The learned GP regression models give a probabilistic prediction of regional activities at next time interval based on observations at present. An anomaly will be detected by comparing the actual observations with the prediction at real time. We verify the effectiveness and robustness of the proposed model on the QMUL Junction Dataset. Furthermore, we provide a publicly available manually labeled ground truth of this data set.

Land cover and land use exhibit strong contextual dependencies. We propose a novel approach for the simultaneous classification of land cover and land use, where semantic and spatial context is considered. The image sites for land cover and land use classification form a hierarchy consisting of two layers: a land cover layer and a land use layer. We apply Conditional Random Fields (CRF) at both layers. The layers differ with respect to the image entities corresponding to the nodes, the employed features and the classes to be distinguished. In the land cover layer, the nodes represent super-pixels; in the land use layer, the nodes correspond to objects from a geospatial database. Both CRFs model spatial dependencies between neighbouring image sites. The complex semantic relations between land cover and land use are integrated in the classification process by using contextual features. We propose a new iterative inference procedure for the simultaneous classification of land cover and land use, in which the two classification tasks mutually influence each other. This helps to improve the classification accuracy for certain classes. The main idea of this approach is that semantic context helps to refine the class predictions, which, in turn, leads to more expressive context information. Thus, potentially wrong decisions can be reversed at later stages. The approach is designed for input data based on aerial images. Experiments are carried out on a test site to evaluate the performance of the proposed method. We show the effectiveness of the iterative inference procedure and demonstrate that a smaller size of the super-pixels has a positive influence on the classification result.

The precise classification and reconstruction of crossroads from multiple aerial images is a challenging problem in remote sensing. We apply the Conditional Random Fields (CRF) approach to this problem, a probabilistic model that can be used to consider context in classification. A simple appearance-based model is combined with a probabilistic model of the co-occurrence of class label at neighbouring image sites to distinguish classes that are relevant for scenes containing crossroads. The parameters of these models are learnt from training data. We use multiple overlap aerial images to derive a digital surface model (DSM) and a true orthophoto without moving cars. From the DSM and the orthophoto we derive feature vectors that are used in the classification. Within our framework we make use of a car detector based on support vector machines (SVM), which delivers car probability values. These values are used as additional feature to support the classification when the road surface is occluded by static cars. Our approach is evaluated on a dataset of airborne photos of an urban area by a comparison of the results to reference data. The evaluation is performed for images of different resolution. The method is shown to produce promising results when using the car probability values and higher image resolution.

The task of crop type classification with multitemporal imagery is nowadays often done applying classifiers that are originally developed for single images like support vector machines (SVM). These approaches do not model temporal dependencies in an explicit way. Existing approaches that make use of temporal dependencies are in most cases quite simple and based on rules. Approaches that integrate temporal dependencies to statistical models are very rare and at an early stage of development. Here our approach CRFmulti, based on conditional random fields (CRF), should make a contribution. Conditional random fields consider context knowledge among neighboring primitives in the same way as Markov random fields (MRF) do. Furthermore conditional random fields handle the feature vectors of the neighboring primitives and not only the class labels. Additional to taking spatial context into account, we present an approach for multitemporal data processing where a temporal association potential has been integrated to the common CRF approach to model temporal dependencies. The classification works on pixel ‐level using spectral image features, whereas all available single images are taken separately. For our experiments a high resolution RapidEye satellite data set of 2010 consisting of 4 images made during the whole vegetation period from April to October is taken. Six crop type categories are distinguished, namely grassland, corn, winter crop, rapeseed, root crops and other crops. To evaluate the potential of the new conditional random field approach the classification result is compared to a manual reference on pixel‐ and on object‐level. Additional a SVM approach is applied under the same conditions and should serve as a benchmark.

Coastal areas are characterized by high spatial and temporal variability. In order to detect undesired changes at early stages, enabling rapid countermeasures to mitigate or minimize potential harm or hazard, a recurrent monitoring becomes necessary. In this paper, we focus on two monitoring task: the analysis of morphological changes and the classification and mapping of habitats. Our concepts are solely based on airborne lidar data which provide substantial information in coastal areas. For the first task, we generate a digital terrain model (DTM) from the lidar point cloud and analyse the dynamic of an island by comparing the DTMs of different epochs with a time difference of six years. For the deeper understanding of the habitat composition in coastal areas, we classify the lidar point cloud by a supervised approach based on Conditional Random Fields. From the classified point cloud, water-land-boundaries as well as mussel bed objects are derived afterwards. We evaluate our approaches on two datasets of the German Wadden Sea.

Advances in unmanned aerial vehicles (UAV) and especially micro aerial vehicle (MAV) technology together with increasing quality and decreasing price of imaging devices have resulted in growing use of MAVs in photogrammetry. The practicality of MAV mapping is seriously enhanced with the ability to determine parameters of exterior orientation (EO) with sufficient accuracy, in both absolute and relative senses (change of attitude between successive images). While differential carrier phase GNSS satisfies cm-level positioning accuracy, precise attitude determination is essential for both direct sensor orientation (DiSO) and integrated sensor orientation (ISO) in corridor mapping or in block configuration imaging over surfaces with low texture. Limited cost, size, and weight of MAVs represent limitations on quality of onboard navigation sensors and puts emphasis on exploiting full capacity of available resources. Typically short flying times (10-30 minutes) also limit the possibility of estimating and/or correcting factors such as sensor misalignment and poor attitude initialization of inertial navigation system (INS). This research aims at increasing the accuracy of attitude determination in both absolute and relative senses with no extra sensors onboard. In comparison to classical INS/GNSS setup, novel approach is presented here to integrated state estimation, in which vehicle dynamic model (VDM) is used as the main process model. Such system benefits from available information from autopilot and physical properties of the platform in enhancing performance of determination of trajectory and parameters of exterior orientation consequently. The navigation system employs a differential carrier phase GNSS receiver and a micro electro-mechanical system (MEMS) grade inertial measurement unit (IMU), together with MAV control input from autopilot. Monte-Carlo simulation has been performed on trajectories for typical corridor mapping and block imaging. Results reveal considerable reduction in attitude errors with respect to conventional INS/GNSS system, in both absolute and relative senses. This eventually translates into higher redundancy and accuracy for photogrammetry applications.

This paper compares two generic approaches for the reconstruction of buildings. Synthesized and real oblique and vertical aerial imagery is transformed on the one hand into a dense photogrammetric 3D point cloud and on the other hand into photogrammetric 2.5D surface models depicting a scene from different cardinal directions. One approach evaluates the 3D point cloud statistically in order to extract the hull of structures, while the other approach makes use of salient line segments in 2.5D surface models, so that the hull of 3D structures can be recovered. With orders of magnitudes more analyzed 3D points, the point cloud based approach is an order of magnitude more accurate for the synthetic dataset compared to the lower dimensioned, but therefor orders of magnitude faster, image processing based approach. For real world data the difference in accuracy between both approaches is not significant anymore. In both cases the reconstructed polyhedra supply information about their inherent semantic and can be used for subsequent and more differentiated semantic annotations through exploitation of texture information.

This study highlights the benefits of precise aerial position and attitude control in the context of mapping with Micro Aerial Vehicles (MAVs). Accurate mapping with MAVs is gaining importance in applications such as corridor mapping, road and pipeline inspections or mapping of large areas with homogeneous surface structure, e.g. forests or agricultural fields. There, accurate aerial control plays a major role in successful terrain reconstruction and artifact-free ortophoto generation. The presented experiments focus on new approaches of aerial control. We confirm practically that the relative aerial position and attitude control can improve accuracy in difficult mapping scenarios. Indeed, the relative orientation method represents an attractive alternative in the context of MAVs for two reasons. First, the procedure is somewhat simplified, e.g. the angular misalignment, so called boresight, between the camera and the inertial measurement unit (IMU) does not have to be determined and, second, the effect of possible systematic errors in satellite positioning (e.g. due to multipath and/or incorrect recovery of differential carrier-phase ambiguities) is mitigated. First, we present a typical mapping project over an agricultural field and second, we perform a corridor road mapping. We evaluate the proposed methods in scenarios with and without automated image observations. We investigate a recently proposed concept where adjustment is performed using image observations limited to ground control and check points, so called fast aerial triangulation (Fast AT). In this context we show that accurate aerial control (absolute or relative) together with a few image observations can deliver accurate results comparable to classical aerial triangulation with thousands of image measurements. This procedure in turns reduces the demands on processing time and the requirements on the existence of surface texture. Finally, we compare the above mentioned procedures with direct sensor orientation (DiSO) to show its potential for rapid mapping.

We describe an effective and efficient method for point-wise semantic classification of 3D point clouds. The method can handle unstructured and inhomogeneous point clouds such as those derived from static terrestrial LiDAR or photogammetric reconstruction; and it is computationally efficient, making it possible to process point clouds with many millions of points in a matter of minutes. The key issue, both to cope with strong variations in point density and to bring down computation time, turns out to be careful handling of neighborhood relations. By choosing appropriate definitions of a point’s (multi-scale) neighborhood, we obtain a feature set that is both expressive and fast to compute. We evaluate our classification method both on benchmark data from a mobile mapping platform and on a variety of large, terrestrial laser scans with greatly varying point density. The proposed feature set outperforms the state of the art with respect to per-point classification accuracy, while at the same time being much faster to compute., ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, III-3, ISSN:2194-9042, ISSN:2194-9050

Previous studies have demonstrated the unprecedented absolute pixel localization accuracy of the German SAR (Synthetic Aperture Radar) satellites TerraSAR-X and TanDEM-X. Now, using global navigation satellite system data to locally correct for atmospheric delay, range accuracies of 1 cm are demonstrated to be attainable. Using globally available tropospheric and ionospheric delay information such accuracies can even be approached world-wide. When such accurate estimates are available for two or more SAR images from different orbits, techniques such as stereo SAR can be used to generate DEMs (Digital Elevation Models) using only the amplitude signal, completely bypassing interferometry. The first experiments of this type described here show that an absolute 3-D localization accuracy better than 10 centimeters is achievable.

ISPRS Hannover Workshop on High-Resolution Earth Imaging for Geospatial Information -- JUN 14-17, 2011 -- Hannover, GERMANY, WOS: 000358235900049, Data collection for digital elevation model (DEM) generation can be carried out by two main methods in space-borne remote sensing such as stereoscopy using optical or radar satellite imagery (stereophotogrammetry, respectively radargrammetry) and interferometry based on interferometric synthetic aperture radar (InSAR) data. These techniques have advantages and disadvantages in comparison against each other. Especially filling the gaps which arise from the problem of cloud coverage in DEM generation by optical imagery, InSAR became operational in recent years and DEMs became the most demanded interferometric products. Essentially, in comparison, DEM generation from synthetic aperture radar (SAR) images is not a simple manner like generation from optical satellite imagery. Interferometric processing has several complicated steps for the production of a DEM. The quality of the data set and used software package come into prominence for the stability of the generated DEM. In the paper, the interferometric processing steps for DEM generation from InSAR data and the crucial threshold values are tried to be explained. For DEM generation, a part of Istanbul (historical peninsula and near surroundings) was selected as the test field because of data availability. The data sets of two different imaging modes (StripMap similar to 3 m resolution and High Resolution Spotlight similar to 1 m resolution) of TerraSAR-X have been used. At the implementation, besides the determination of crucial points at interferometric processing steps, to define the effect of computer software, DEM production have been performed using two different software packages in parallel and the products have been compared In the result section of the paper, besides the colorful visualizations of final products along with the height scales, accuracy evaluations have been performed for both DEMs with the help of a more accurate reference digital terrain model (DTM). This reference model has been achieved by large scale aerial photos. Normally, it has a 5 m original grid spacing, however it has been resampled at a spacing of 1 m towards the needs of the research., Int Soc Photogrammetry & Remote Sensing

ISPRS Hannover Workshop on High-Resolution Earth Imaging for Geospatial Information -- JUN 14-17, 2011 -- Hannover, GERMANY, WOS: 000358235900001, Nowadays Geographic Information Systems (GIS) uses Remote Sensing (RS) data for a lot of applications. One of the application areas is the updating of the GIS database using high resolution imagery. In this context high resolution satellite imagery data is very important for many applications areas today's and future. And also, high resolution satellite imagery data will be used in many applications for different purposes. Information systems needs to high resolution imagery data for updating. Updating is very important component for the any of the GIS systems. One of this area will be updated and kept alive GIS database information. High resolution satellite imagery is used with different data base which serve map information via internet and different aims of information systems applications in future topographic and cartographic information systems will very important in our country in this sense use of the satellite images will be unavoidable. In this study explain to how is acquired to satellite images and how is use this images in information systems for object and roads. Firstly, pan-sharpened two of the IKONOS's images have been produced by fusion of high resolution PAN and MS images using PCI Geomatica v9.1 software package. Automatic object extraction has been made using eCognition v4.0.6. On the other hand, these objects have been manually digitized from high resolution images using ArcGIS v9.3. software package. Application section of in this study, satellite images data will be compared each other and GIS objects and road database. It is also determined which data is useful in Geographic Information Systems. Finally, this article explains that integration of remote sensing technology and GIS applications., Int Soc Photogrammetry & Remote Sensing

Data fusion techniques require a good registration of all the used datasets. In remote sensing, images are usually geo-referenced using the GPS and IMU data. However, if more precise registration is required, image processing techniques can be employed. We propose a method for multi-modal image coregistration between hyperspectral images (HSI) and digital surface models (DSM). The method is divided in three parts: object and line detection of the same object in HSI and DSM, line matching and determination of transformation parameters. Homogeneous coordinates are used to implement matching and adjustment of transformation parameters. The common object in HSI and DSM are building boundaries. They have apparent change in height and material, that can be detected in DSM and HSI, respectively. Thus, before the matching and transformation parameter computation, building outlines are detected and adjusted in HSI and DSM. We test the method on a HSI and two DSM, using extracted building outbounds and for comparison also extracted lines with a line detector. The results show that estimated building boundaries provide more line assignments, than using line detector.