Automatic reconstruction of 3D models from images using multi-view Structure-from-Motion methods ... more Automatic reconstruction of 3D models from images using multi-view Structure-from-Motion methods has been one of the most fruitful outcomes of computer vision. These advances combined with the growing popularity of Micro Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools ubiquitous for large number of Architecture, Engineering and Construction applications among audiences, mostly unskilled in computer vision. However, to obtain high-resolution and accurate reconstructions from a large-scale object using SfM, there are many critical constraints on the quality of image data, which often become sources of inaccuracy as the current 3D reconstruction pipelines do not facilitate the users to determine the fidelity of input data during the image acquisition. In this paper, we present and advocate a closed-loop interactive approach that performs incremental reconstruction in real-time and gives users an online feedback about the quality parameters like Ground Sampl...
This paper presents and discusses algorithms, hardware, and software architecture developed by th... more This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved 2nd and 1st place, respectively. We also discuss CoSTAR’s demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including: (i) geometric and semantic environment mapping; (ii) a multi-modal positioning system; (iii) traversability...
Signal Processing, Pattern Recognition and Applications / 779: Computer Graphics and Imaging, 2012
ABSTRACT Fingerprint verification is an important biometric technique for personal identification... more ABSTRACT Fingerprint verification is an important biometric technique for personal identification. This paper presents a performance evaluation of different fingerprint feature extraction methods. Fingerprint matching scheme based on transform features, like DCT (Discrete Cosine Transform), FFT (Fast Fourier Transform) and DWT (Discrete wavelet transform), have been presented and compared. In the fingerprint recognition application utilizing more information other than minutiae is much helpful. The technique described here obviates the need for extracting minutiae points to match fingerprint images. The transform coefficients are used to obtain the feature vector in terms of standard deviation and energy. Matching is performed using fast Euclidean distance between two feature vectors. The algorithms have been tested on database available from University of Bologna. Our comparison shows that DCT and FFT yields better GAR (Genuine acceptance rate) at low FAR (False acceptance rate) with reduced computational complexity over existing DWT and Gabor based algorithms. Because of reduced computational complexity these algorithms can be easily implemented as an embedded automatic fingerprint identification system (AFIS).
ABSTRACT This paper focuses on the design and development of a pipe inspection robot and quantifi... more ABSTRACT This paper focuses on the design and development of a pipe inspection robot and quantification of the usability of a Polyvinylidene fluoride (PVDF) based cantilever smart probe technique for detection, localization and sizing of surface defects. The pipe crawler robot, controlled by a remote desktop through a wireless communication system, can go inside the pipelines and make inspection automatically to scan its surface quality. The smart probe, during rotation, touches the inner surface of the pipe and experience a broad-band excitation in the absence of surface features. On the other hand, whenever the probe comes across any surface defect, there is a change in vibration response of the probe. The discriminating time domain features of the observed damping waveform such as damping factor, peak value and settling time are used to interpret the experimental results with the defects in the test sample. The sensor system has reliably predicted the presence and distribution of projections, holes and surface roughness artificially constructed inside the pipe in each case. It is projected that the new sensing system could be used efficiently for pipe health monitoring.
Onboard localization capabilities for planetary rovers to date have used relative navigation, by ... more Onboard localization capabilities for planetary rovers to date have used relative navigation, by integrating combinations of wheel odometry, visual odometry, and inertial measurements during each drive to track position relative to the start of each drive. At the end of each drive, a ground-in-the-loop (GITL) interaction is used to get a position update from human operators in a more global reference frame, by matching images or local maps from onboard the rover to orbital reconnaissance images or maps of a large region around the rover's current position. Autonomous rover drives are limited in distance so that accumulated relative navigation error does not risk the possibility of the rover driving into hazards known from orbital images. However, several rover mission concepts have recently been studied that require much longer drives between GITL cycles, particularly for the Moon. These concepts require greater autonomy to minimize GITL cycles to enable such large range; onboard global localization is a key element of such autonomy. Multiple techniques have been studied in the past for onboard rover global localization, but a satisfactory solution has not yet emerged. For the Moon, the ubiquitous craters offer a new possibility, which involves mapping craters from orbit, then recognizing crater landmarks with cameras and-or a lidar onboard the rover. This approach is applicable everywhere on the Moon, does not require high resolution stereo imaging from orbit as some other approaches do, and has potential to enable position knowledge with order of 5 to 10 m accuracy at all times. This paper describes our technical approach to crater-based lunar rover localization and presents initial results on crater detection using 3D point cloud data from onboard lidar or stereo cameras, as well as using shading cues in monocular onboard imagery.
Thermal infrared cameras are increasingly being used in various applications such as robot vision... more Thermal infrared cameras are increasingly being used in various applications such as robot vision, industrial inspection and medical imaging, thanks to their improved resolution and portability. However, the performance of traditional computer vision techniques developed for electro-optical imagery does not directly translate to the thermal domain due to two major reasons: these algorithms require photometric assumptions to hold, and methods for photometric calibration of RGB cameras cannot be applied to thermal-infrared cameras due to difference in data acquisition and sensor phenomenology. In this paper, we take a step in this direction, and introduce a novel algorithm for online photometric calibration of thermal-infrared cameras. Our proposed method does not require any specific driver/hardware support and hence can be applied to any commercial off-the-shelf thermal IR camera. We present this in the context of visual odometry and SLAM algorithms, and demonstrate the efficacy of ...
Photogrammetric computer vision systems have been well established in many scientific and commerc... more Photogrammetric computer vision systems have been well established in many scientific and commercial fields during the last decades. Recent developments in image-based 3D reconstruction systems in conjunction with the availability of affordable high quality digital consumer grade cameras have resulted in an easy way of creating visually appealing 3D models. However, many of these methods require manual steps in the processing chain and for many photogrammetric applications such as mapping, recurrent topographic surveys or architectural and archaeological 3D documentations, high accuracy in a geo-coordinate system is required which often cannot be guaranteed. Hence, in this paper we present and advocate a fully automated end-to-end workflow for precise and geo-accurate 3D reconstructions using fiducial markers. We integrate an automatic camera calibration and georeferencing method into our image-based reconstruction pipeline based on binary-coded fiducial markers as artificial, indiv...
The benefit of accurate camera calibration for recovering 3D structure from images is a well-stud... more The benefit of accurate camera calibration for recovering 3D structure from images is a well-studied topic. Recently 3D vision tools for end-user applications have become popular among large audiences, mostly unskilled in computer vision. This motivates the need for a flexible and user-centric camera calibration method which drastically releases the critical requirements on the calibration target and ensures that low-quality or faulty images provided by end users do not degrade the overall calibration and in effect the resulting 3D model. In this paper we present and advocate an approach to camera cal-ibration using fiducial markers, aiming at the accuracy of target calibration techniques without the requirement for a precise calibration pattern, to ease the calibration effort for the end-user. An extensive set of experiments with real images is presented which demonstrates improvements in the estimation of the parameters of the camera model as well as accuracy in the multi-view ste...
Abstract—Automatic reconstruction of 3D models from im-ages using multi-view Structure-from-Motio... more Abstract—Automatic reconstruction of 3D models from im-ages using multi-view Structure-from-Motion methods has been one of the most fruitful outcomes of computer vision. These advances combined with the growing popularity of Micro Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools ubiquitous for large number of Architecture, Engineering and Construction applications among audiences, mostly unskilled in computer vision. However, to obtain high-resolution and accurate reconstructions from a large-scale object using SfM, there are many critical constraints on the quality of image data, which often become sources of inaccuracy as the current 3D reconstruction pipelines do not facilitate the users to determine the fidelity of input data during the image acquisition. In this paper, we present and advocate a closed-loop interactive approach that performs incremental reconstruction in real-time and gives users an online feedback about the quality parameters like Gr...
This paper studies the problem of steering a linear time-invariant system subject to state and in... more This paper studies the problem of steering a linear time-invariant system subject to state and input constraints towards a goal location that may be inferred only through partial observations. We assume mixed-observable settings, where the system's state is fully observable and the environment's state defining the goal location is only partially observed. In these settings, the planning problem is an infinite-dimensional optimization problem where the objective is to minimize the expected cost. We show how to reformulate the control problem as a finite-dimensional deterministic problem by optimizing over a trajectory tree. Leveraging this result, we demonstrate that when the environment is static, the observation model piecewise, and cost function convex, the original control problem can be reformulated as a Mixed-Integer Convex Program (MICP) that can be solved to global optimality using a branch-and-bound algorithm. The effectiveness of the proposed approach is demonstrate...
Iris recognition is one of the challenging problems inhuman computer interaction. An automated ir... more Iris recognition is one of the challenging problems inhuman computer interaction. An automated iris recognition system requires an efficient method for classification of iris region in the face sequence, extraction of iris features, and construction of classification model. In recent years, Neural Networks (NN) has demonstrated excellent performance in a variety of classification problems. In this paper, we have used a simple 2dimensionaldiscrete wavelet transform (DWT) representation which captures the small differences in the image that is desired for the current applications. The DWT is used to generate feature images from individual wavelet sub bands. The results of our studies show that, the system gives about 90.00%recognition rate.
A potential Mars Sample Return (MSR) architecture is being jointly studied by NASA and ESA. As cu... more A potential Mars Sample Return (MSR) architecture is being jointly studied by NASA and ESA. As currently envisioned, the MSR campaign consists of a series of 3 missions: sample cache, fetch and return to Earth. In this paper, we focus on the fetch part of the MSR, and more specifically the problem of autonomously detecting and localizing sample tubes deposited on the Martian surface. Towards this end, we study two machine-vision based approaches: First, a geometry-driven approach based on template matching that uses hard-coded filters and a 3D shape model of the tube; and second, a data-driven approach based on convolutional neural networks (CNNs) and learned features. Furthermore, we present a large benchmark dataset of sample-tube images, collected in representative outdoor environments and annotated with ground truth segmentation masks and locations. The dataset was acquired systematically across different terrain, illumination conditions and dust-coverage; and benchmarking was p...
Titan's dense atmosphere, low gravity, and high winds at high altitudes create descent times ... more Titan's dense atmosphere, low gravity, and high winds at high altitudes create descent times of >90 minutes with standard entry/descent/landing (EDL) architectures and result in large unguided landing ellipses, with 99% values of ~110x110 km and 149x72 km in recent Titan lander proposals. Enabling precision landing on Titan could increase science return for the types of missions proposed to date and make additional types of landing sites accessible, opening up new possibilities for science investigations. Precision landing on Titan has unique challenges, because the hazy atmosphere makes it difficult to see the surface and because it requires guided descent with divert ranges that are one to two orders of magnitude larger than needed for other target bodies, i.e. up to on the order of 100 km. It is conceivable that such a divert capability could be provided economically by a parafoil or other steerable aerodynamic decelerator deployed several 10s of km above the surface. The ...
Enhanced AutoNav (ENav), the baseline surface navigation software for NASA's Perseverance rov... more Enhanced AutoNav (ENav), the baseline surface navigation software for NASA's Perseverance rover, sorts a list of candidate paths for the rover to traverse, then uses the Approximate Clearance Evaluation (ACE) algorithm to evaluate whether the most highly ranked paths are safe. ACE is crucial for maintaining the safety of the rover, but is computationally expensive. If the most promising candidates in the list of paths are all found to be infeasible, ENav must continue to search the list and run time-consuming ACE evaluations until a feasible path is found. In this paper, we present two heuristics that, given a terrain heightmap around the rover, produce cost estimates that more effectively rank the candidate paths before ACE evaluation. The first heuristic uses Sobel operators and convolution to incorporate the cost of traversing high-gradient terrain. The second heuristic uses a machine learning (ML) model to predict areas that will be deemed untraversable by ACE. We used physi...
Automatic reconstruction of 3D models from images using multi-view Structure-from-Motion methods ... more Automatic reconstruction of 3D models from images using multi-view Structure-from-Motion methods has been one of the most fruitful outcomes of computer vision. These advances combined with the growing popularity of Micro Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools ubiquitous for large number of Architecture, Engineering and Construction applications among audiences, mostly unskilled in computer vision. However, to obtain high-resolution and accurate reconstructions from a large-scale object using SfM, there are many critical constraints on the quality of image data, which often become sources of inaccuracy as the current 3D reconstruction pipelines do not facilitate the users to determine the fidelity of input data during the image acquisition. In this paper, we present and advocate a closed-loop interactive approach that performs incremental reconstruction in real-time and gives users an online feedback about the quality parameters like Ground Sampl...
This paper presents and discusses algorithms, hardware, and software architecture developed by th... more This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved 2nd and 1st place, respectively. We also discuss CoSTAR’s demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including: (i) geometric and semantic environment mapping; (ii) a multi-modal positioning system; (iii) traversability...
Signal Processing, Pattern Recognition and Applications / 779: Computer Graphics and Imaging, 2012
ABSTRACT Fingerprint verification is an important biometric technique for personal identification... more ABSTRACT Fingerprint verification is an important biometric technique for personal identification. This paper presents a performance evaluation of different fingerprint feature extraction methods. Fingerprint matching scheme based on transform features, like DCT (Discrete Cosine Transform), FFT (Fast Fourier Transform) and DWT (Discrete wavelet transform), have been presented and compared. In the fingerprint recognition application utilizing more information other than minutiae is much helpful. The technique described here obviates the need for extracting minutiae points to match fingerprint images. The transform coefficients are used to obtain the feature vector in terms of standard deviation and energy. Matching is performed using fast Euclidean distance between two feature vectors. The algorithms have been tested on database available from University of Bologna. Our comparison shows that DCT and FFT yields better GAR (Genuine acceptance rate) at low FAR (False acceptance rate) with reduced computational complexity over existing DWT and Gabor based algorithms. Because of reduced computational complexity these algorithms can be easily implemented as an embedded automatic fingerprint identification system (AFIS).
ABSTRACT This paper focuses on the design and development of a pipe inspection robot and quantifi... more ABSTRACT This paper focuses on the design and development of a pipe inspection robot and quantification of the usability of a Polyvinylidene fluoride (PVDF) based cantilever smart probe technique for detection, localization and sizing of surface defects. The pipe crawler robot, controlled by a remote desktop through a wireless communication system, can go inside the pipelines and make inspection automatically to scan its surface quality. The smart probe, during rotation, touches the inner surface of the pipe and experience a broad-band excitation in the absence of surface features. On the other hand, whenever the probe comes across any surface defect, there is a change in vibration response of the probe. The discriminating time domain features of the observed damping waveform such as damping factor, peak value and settling time are used to interpret the experimental results with the defects in the test sample. The sensor system has reliably predicted the presence and distribution of projections, holes and surface roughness artificially constructed inside the pipe in each case. It is projected that the new sensing system could be used efficiently for pipe health monitoring.
Onboard localization capabilities for planetary rovers to date have used relative navigation, by ... more Onboard localization capabilities for planetary rovers to date have used relative navigation, by integrating combinations of wheel odometry, visual odometry, and inertial measurements during each drive to track position relative to the start of each drive. At the end of each drive, a ground-in-the-loop (GITL) interaction is used to get a position update from human operators in a more global reference frame, by matching images or local maps from onboard the rover to orbital reconnaissance images or maps of a large region around the rover's current position. Autonomous rover drives are limited in distance so that accumulated relative navigation error does not risk the possibility of the rover driving into hazards known from orbital images. However, several rover mission concepts have recently been studied that require much longer drives between GITL cycles, particularly for the Moon. These concepts require greater autonomy to minimize GITL cycles to enable such large range; onboard global localization is a key element of such autonomy. Multiple techniques have been studied in the past for onboard rover global localization, but a satisfactory solution has not yet emerged. For the Moon, the ubiquitous craters offer a new possibility, which involves mapping craters from orbit, then recognizing crater landmarks with cameras and-or a lidar onboard the rover. This approach is applicable everywhere on the Moon, does not require high resolution stereo imaging from orbit as some other approaches do, and has potential to enable position knowledge with order of 5 to 10 m accuracy at all times. This paper describes our technical approach to crater-based lunar rover localization and presents initial results on crater detection using 3D point cloud data from onboard lidar or stereo cameras, as well as using shading cues in monocular onboard imagery.
Thermal infrared cameras are increasingly being used in various applications such as robot vision... more Thermal infrared cameras are increasingly being used in various applications such as robot vision, industrial inspection and medical imaging, thanks to their improved resolution and portability. However, the performance of traditional computer vision techniques developed for electro-optical imagery does not directly translate to the thermal domain due to two major reasons: these algorithms require photometric assumptions to hold, and methods for photometric calibration of RGB cameras cannot be applied to thermal-infrared cameras due to difference in data acquisition and sensor phenomenology. In this paper, we take a step in this direction, and introduce a novel algorithm for online photometric calibration of thermal-infrared cameras. Our proposed method does not require any specific driver/hardware support and hence can be applied to any commercial off-the-shelf thermal IR camera. We present this in the context of visual odometry and SLAM algorithms, and demonstrate the efficacy of ...
Photogrammetric computer vision systems have been well established in many scientific and commerc... more Photogrammetric computer vision systems have been well established in many scientific and commercial fields during the last decades. Recent developments in image-based 3D reconstruction systems in conjunction with the availability of affordable high quality digital consumer grade cameras have resulted in an easy way of creating visually appealing 3D models. However, many of these methods require manual steps in the processing chain and for many photogrammetric applications such as mapping, recurrent topographic surveys or architectural and archaeological 3D documentations, high accuracy in a geo-coordinate system is required which often cannot be guaranteed. Hence, in this paper we present and advocate a fully automated end-to-end workflow for precise and geo-accurate 3D reconstructions using fiducial markers. We integrate an automatic camera calibration and georeferencing method into our image-based reconstruction pipeline based on binary-coded fiducial markers as artificial, indiv...
The benefit of accurate camera calibration for recovering 3D structure from images is a well-stud... more The benefit of accurate camera calibration for recovering 3D structure from images is a well-studied topic. Recently 3D vision tools for end-user applications have become popular among large audiences, mostly unskilled in computer vision. This motivates the need for a flexible and user-centric camera calibration method which drastically releases the critical requirements on the calibration target and ensures that low-quality or faulty images provided by end users do not degrade the overall calibration and in effect the resulting 3D model. In this paper we present and advocate an approach to camera cal-ibration using fiducial markers, aiming at the accuracy of target calibration techniques without the requirement for a precise calibration pattern, to ease the calibration effort for the end-user. An extensive set of experiments with real images is presented which demonstrates improvements in the estimation of the parameters of the camera model as well as accuracy in the multi-view ste...
Abstract—Automatic reconstruction of 3D models from im-ages using multi-view Structure-from-Motio... more Abstract—Automatic reconstruction of 3D models from im-ages using multi-view Structure-from-Motion methods has been one of the most fruitful outcomes of computer vision. These advances combined with the growing popularity of Micro Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools ubiquitous for large number of Architecture, Engineering and Construction applications among audiences, mostly unskilled in computer vision. However, to obtain high-resolution and accurate reconstructions from a large-scale object using SfM, there are many critical constraints on the quality of image data, which often become sources of inaccuracy as the current 3D reconstruction pipelines do not facilitate the users to determine the fidelity of input data during the image acquisition. In this paper, we present and advocate a closed-loop interactive approach that performs incremental reconstruction in real-time and gives users an online feedback about the quality parameters like Gr...
This paper studies the problem of steering a linear time-invariant system subject to state and in... more This paper studies the problem of steering a linear time-invariant system subject to state and input constraints towards a goal location that may be inferred only through partial observations. We assume mixed-observable settings, where the system's state is fully observable and the environment's state defining the goal location is only partially observed. In these settings, the planning problem is an infinite-dimensional optimization problem where the objective is to minimize the expected cost. We show how to reformulate the control problem as a finite-dimensional deterministic problem by optimizing over a trajectory tree. Leveraging this result, we demonstrate that when the environment is static, the observation model piecewise, and cost function convex, the original control problem can be reformulated as a Mixed-Integer Convex Program (MICP) that can be solved to global optimality using a branch-and-bound algorithm. The effectiveness of the proposed approach is demonstrate...
Iris recognition is one of the challenging problems inhuman computer interaction. An automated ir... more Iris recognition is one of the challenging problems inhuman computer interaction. An automated iris recognition system requires an efficient method for classification of iris region in the face sequence, extraction of iris features, and construction of classification model. In recent years, Neural Networks (NN) has demonstrated excellent performance in a variety of classification problems. In this paper, we have used a simple 2dimensionaldiscrete wavelet transform (DWT) representation which captures the small differences in the image that is desired for the current applications. The DWT is used to generate feature images from individual wavelet sub bands. The results of our studies show that, the system gives about 90.00%recognition rate.
A potential Mars Sample Return (MSR) architecture is being jointly studied by NASA and ESA. As cu... more A potential Mars Sample Return (MSR) architecture is being jointly studied by NASA and ESA. As currently envisioned, the MSR campaign consists of a series of 3 missions: sample cache, fetch and return to Earth. In this paper, we focus on the fetch part of the MSR, and more specifically the problem of autonomously detecting and localizing sample tubes deposited on the Martian surface. Towards this end, we study two machine-vision based approaches: First, a geometry-driven approach based on template matching that uses hard-coded filters and a 3D shape model of the tube; and second, a data-driven approach based on convolutional neural networks (CNNs) and learned features. Furthermore, we present a large benchmark dataset of sample-tube images, collected in representative outdoor environments and annotated with ground truth segmentation masks and locations. The dataset was acquired systematically across different terrain, illumination conditions and dust-coverage; and benchmarking was p...
Titan's dense atmosphere, low gravity, and high winds at high altitudes create descent times ... more Titan's dense atmosphere, low gravity, and high winds at high altitudes create descent times of >90 minutes with standard entry/descent/landing (EDL) architectures and result in large unguided landing ellipses, with 99% values of ~110x110 km and 149x72 km in recent Titan lander proposals. Enabling precision landing on Titan could increase science return for the types of missions proposed to date and make additional types of landing sites accessible, opening up new possibilities for science investigations. Precision landing on Titan has unique challenges, because the hazy atmosphere makes it difficult to see the surface and because it requires guided descent with divert ranges that are one to two orders of magnitude larger than needed for other target bodies, i.e. up to on the order of 100 km. It is conceivable that such a divert capability could be provided economically by a parafoil or other steerable aerodynamic decelerator deployed several 10s of km above the surface. The ...
Enhanced AutoNav (ENav), the baseline surface navigation software for NASA's Perseverance rov... more Enhanced AutoNav (ENav), the baseline surface navigation software for NASA's Perseverance rover, sorts a list of candidate paths for the rover to traverse, then uses the Approximate Clearance Evaluation (ACE) algorithm to evaluate whether the most highly ranked paths are safe. ACE is crucial for maintaining the safety of the rover, but is computationally expensive. If the most promising candidates in the list of paths are all found to be infeasible, ENav must continue to search the list and run time-consuming ACE evaluations until a feasible path is found. In this paper, we present two heuristics that, given a terrain heightmap around the rover, produce cost estimates that more effectively rank the candidate paths before ACE evaluation. The first heuristic uses Sobel operators and convolution to incorporate the cost of traversing high-gradient terrain. The second heuristic uses a machine learning (ML) model to predict areas that will be deemed untraversable by ACE. We used physi...
Uploads
Papers