Search Results (81)

This study presents an integrated system for object recognition, six-degrees-of-freedom pose estimation, and dexterous manipulation using a JACO robotic arm with an Intel RealSense D435 camera. This system is designed to automate the manipulation of industrial valves by capturing point clouds (PCs) from multiple perspectives to improve the accuracy of pose estimation. The object recognition module includes scene segmentation, geometric primitives recognition, model recognition, and a color-based clustering and integration approach enhanced by a dynamic cluster merging algorithm. Pose estimation is achieved using the random sample consensus algorithm, which predicts position and orientation. The system was tested within a 60° field of view, which extended in all directions in front of the object. The experimental results show that the system performs reliably within acceptable error thresholds for both position and orientation when the objects are within a ±15° range of the camera’s direct view. However, errors increased with more extreme object orientations and distances, particularly when estimating the orientation of ball valves. A zone-based dexterous manipulation strategy was developed to overcome these challenges, where the system adjusts the camera position for optimal conditions. This approach mitigates larger errors in difficult scenarios, enhancing overall system reliability. The key contributions of this research include a novel method for improving object recognition and pose estimation, a technique for increasing the accuracy of pose estimation, and the development of a robot motion model for dexterous manipulation in industrial settings. Full article

This paper proposes an evaluating and mapping methodology of terrain traversability for off-road navigation of autonomous vehicles in unstructured environments. Terrain features are extracted from RGB images and 3D point clouds to create a traversal cost map. The cost map is then employed to plan safe trajectories. Bayesian generalized kernel inference is employed to assess unknown grid attributes due to the sparse raw point cloud data. A Kalman filter also creates density local elevation maps in real time by fusing multiframe information. Consequently, the terrain semantic mapping procedure considers the uncertainty of semantic segmentation and the impact of sensor noise. A Bayesian filter is used to update the surface semantic information in a probabilistic manner. Ultimately, the elevation map is utilized to extract geometric characteristics, which are then integrated with the probabilistic semantic map. This combined map is then used in conjunction with the extended motion primitive planner to plan the most effective trajectory. The experimental results demonstrate that the autonomous vehicles obtain a success rate enhancement ranging from 4.4% to 13.6% and a decrease in trajectory roughness ranging from 5.1% to 35.8% when compared with the most developed outdoor navigation algorithms. Additionally, the autonomous vehicles maintain a terrain surface selection accuracy of over 85% during the navigation process. Full article

In low-altitude, GNSS-denied scenarios, Unmanned aerial vehicles (UAVs) rely on sensor fusion for self-localization. This article presents a resilient multi-sensor fusion localization system that integrates light detection and ranging (LiDAR), cameras, and inertial measurement units (IMUs) to achieve state estimation for UAVs. To address challenging environments, especially unstructured ones, IMU predictions are used to compensate for pose estimation in the visual and LiDAR components. Specifically, the accuracy of IMU predictions is enhanced by increasing the correction frequency of IMU bias through data integration from the LiDAR and visual modules. To reduce the impact of random errors and measurement noise in LiDAR points on visual depth measurement, cross-validation of visual feature depth is performed using reprojection error to eliminate outliers. Additionally, a structure monitor is introduced to switch operation modes in hybrid point cloud registration, ensuring accurate state estimation in both structured and unstructured environments. In unstructured scenes, a geometric primitive capable of representing irregular planes is employed for point-to-surface registration, along with a novel pose-solving method to estimate the UAV’s pose. Both private and public datasets collected by UAVs validate the proposed system, proving that it outperforms state-of-the-art algorithms by at least 12.6%. Full article

Automated generation of geometric models from point cloud data holds significant importance in the field of computer vision and has expansive applications, such as shape modeling and object recognition. However, prevalent methods exhibit accuracy issues. In this study, we introduce a novel hierarchical neural network that utilizes recursive PointConv operations on nested subdivisions of point sets. This network effectively extracts features, segments point clouds, and accurately identifies and computes parameters of regular geometric primitives with notable resilience to noise. On fine-grained primitive detection, our approach outperforms Supervised Primitive Fitting Network (SPFN) by 18.5% and Cascaded Primitive Fitting Network (CPFN) by 11.2%. Additionally, our approach consistently maintains low absolute errors in parameter prediction across varying noise levels in the point cloud data. Our experiments validate the robustness of our proposed method and establish its superiority relative to other methodologies in the extant literature. Full article

This paper explores the problem of automated robot program generation from limited historical data when neither accurate geometric environmental models nor online vision feedback are available. The Program Reuse Framework (PRF) is developed, which uses expert-defined motion classes, a novel data structure introduced in this work, to learn affordances, workspaces, and skills from historical data. Historical data comprise raw robot joint trajectories and descriptions of the robot task being completed. Given new tasks, motion classes are then used again to formulate an optimization problem capable of generating new open-loop, skill-based programs to complete the tasks. To cope with a lack of geometric models, a technique to learn safe workspaces from demonstrations is developed, allowing the risk of new programs to be estimated before execution. A new learnable motion primitive for redundant manipulators is introduced, called a redundancy dynamical movement primitive, which enables new end-effector goals to be reached while mimicking the whole-arm behavior of a demonstration. A mobile manipulator part transportation task is used throughout to illustrate each step of the framework. Full article

The insufficient generalisation capability of the conventional learning from demonstration (LfD) model necessitates redemonstrations. In addition, retraining the model can overwrite existing knowledge, making it impossible to perform previously acquired skills in new application scenarios. These are not economical and efficient. To address the issues, in this study, a broad learning system (BLS) and probabilistic roadmap (PRM) are integrated with dynamic movement primitive (DMP)-based LfD. Three key innovations are proposed in this paper: (1) segmentation and extended demonstration: a 1D-based topology trajectory segmentation algorithm (1D-SEG) is designed to divide the original demonstration into several segments. Following the segmentation, a Gaussian probabilistic roadmap (G-PRM) is proposed to generate an extended demonstration that retains the geometric features of the original demonstration. (2) DMP modelling and incremental learning updating: BLS-based incremental learning for DMP (Bi-DMP) is performed based on the constructed DMP and extended demonstration. With this incremental learning approach, the DMP is capable of self-updating in response to task demands, preserving previously acquired skills and updating them without training from scratch. (3) Electric vehicle (EV) battery disassembly case study: this study developed a solution suitable for EV battery disassembly and established a decommissioned battery disassembly experimental platform. Unscrewing nuts and battery cell removal are selected to verify the effectiveness of the proposed algorithms based on the battery disassembly experimental platform. In this study, the effectiveness of the algorithms designed in this paper is measured by the success rate and error of the task execution. In the task of unscrewing nuts, the success rate of the classical DMP is 57.14% and the maximum error is 2.760 mm. After the optimisation of 1D-SEG, G-PRM, and Bi-DMP, the success rate of the task is increased to 100% and the maximum error is reduced to 1.477 mm. Full article

Triply periodic minimal surface microarchitectures (TPMS) were developed by mathematicians and evolved in all kingdoms of living organisms. Renowned for their lightweight yet robust attributes, TPMS structures find application in diverse fields, such as the construction of satellites, aircrafts, and electric vehicles. Moreover, these microarchitectures, despite their intricate geometric patterns, demonstrate potential for application as bone substitutes, despite the inherent gothic style of natural bone microarchitecture. Here, we produced three TPMS microarchitectures, D-diamond, G-gyroid, and P-primitive, by 3D printing from hydroxyapatite. We explored their mechanical characterization and, further, implanted them to study their bone augmentation and osteoconduction potential. In terms of strength, the D-diamond and G-gyroid performed significantly better than the P-primitive. In a calvarial defect model and a calvarial bone augmentation model, where osteoconduction is determined as the extent of bony bridging of the defect and bone augmentation as the maximal vertical bone ingrowth, the G-gyroid performed significantly better than the P-primitive. No significant difference in performance was observed between the G-gyroid and D-diamond. Since, in real life, the treatment of bone deficiencies in patients comprises elements of defect bridging and bone augmentation, ceramic scaffolds with D-diamond and G-gyroid microarchitectures appear as the best choice for a TPMS-based scaffold in bone tissue engineering. Full article

Craters are regarded as significant navigation landmarks during the descent and landing process in small body exploration missions for their universality. Recognizing and matching craters is a crucial prerequisite for visual and LIDAR-based navigation tasks. Compared to traditional algorithms, deep learning-based crater detection algorithms can achieve a higher recognition rate. However, matching crater detection results under various image transformations still poses challenges. To address the problem, a composite feature-matching algorithm that combines geometric descriptors and region descriptors (extracting normalized region pixel gradient features as feature vectors) is proposed. First, the geometric configuration map is constructed based on the crater detection results. Then, geometric descriptors and region descriptors are established within each feature primitive of the map. Subsequently, taking the salience of geometric features into consideration, composite feature descriptors with scale, rotation, and illumination invariance are generated through fusion geometric and region descriptors. Finally, descriptor matching is accomplished by computing the relative distances between descriptors and adhering to the nearest neighbor principle. Experimental results show that the composite feature descriptor proposed in this paper has better matching performance than only using shape descriptors or region descriptors, and can achieve a more than 90% correct matching rate, which can provide technical support for the small body visual navigation task. Full article

A wireless capsule endoscope (WCE) is a medical device designed for the examination of the human gastrointestinal (GI) tract. Three-dimensional models based on WCE images can assist in diagnostics by effectively detecting pathology. These 3D models provide gastroenterologists with improved visualization, particularly in areas of specific interest. However, the constraints of WCE, such as lack of controllability, and requiring expensive equipment for operation, which is often unavailable, pose significant challenges when it comes to conducting comprehensive experiments aimed at evaluating the quality of 3D reconstruction from WCE images. In this paper, we employ a single-image-based 3D reconstruction method on an artificial colon captured with an endoscope that behaves like WCE. The shape from shading (SFS) algorithm can reconstruct the 3D shape using a single image. Therefore, it has been employed to reconstruct the 3D shapes of the colon images. The camera of the endoscope has also been subjected to comprehensive geometric and radiometric calibration. Experiments are conducted on well-defined primitive objects to assess the method’s robustness and accuracy. This evaluation involves comparing the reconstructed 3D shapes of primitives with ground truth data, quantified through measurements of root-mean-square error and maximum error. Afterward, the same methodology is applied to recover the geometry of the colon. The results demonstrate that our approach is capable of reconstructing the geometry of the colon captured with a camera with an unknown imaging pipeline and significant noise in the images. The same procedure is applied on WCE images for the purpose of 3D reconstruction. Preliminary results are subsequently generated to illustrate the applicability of our method for reconstructing 3D models from WCE images. Full article

Lattice materials are widely used in industries due to their designable capabilities of specific stiffness and energy absorption. However, evaluating the mechanical response of macroscopic lattice structures can be computationally expensive. Homogenization-based multi-scale analysis offers an efficient approach to address this issue. To achieve a simpler, while precise, homogenization, the authors proposed an equidistant segmentation (ES) method for the measurement of the effective shear modulus. In this method, the periodic boundary conditions (PBCs) are approximated by constraining the lateral displacement of nodes between parallel layers of periodic cells. The validations were applied to three typical lattice topologies: body-centered cubic (BCC) lattices, gyroid-, and primitive-triply periodic minimal surface (TPMS) lattices, to predict and compare their anti-vibration capacities. The results demonstrated the rationality and the promising precision of the multi-scale-based equivalent modal analysis through the proposed method and that it eliminated the geometric limitation of lattices with diverse frameworks. Overall, a higher anti-vibration capacity of TPMS was observed. In the study, the authors examined the influence of the relative densities on the balance between the anti-vibration capacity and loading capacity (per unit mass) of the TPMS topologies. Specifically, the unit mass of the TPMS with lower relative densities was able to resist higher frequencies, and the structures were dominated by the anti-vibration capacity. In contrast, a higher relative density is better when emphasizing the loading capacity. These findings may provide notable references to the designers and inform the selection of lattice materials for various industrial applications. Full article

With the advancement of China’s wind power industry, research into full-scale structural testing of wind turbine blades, including static testing and fatigue testing, has shown increasing significance. Static testing measures the deflection at fixed points, using pull-wire sensors in industrial practice. However, the demerits of this method involve single dimension, excessive deviation, costly experiment, and complex installment. Given the advantages that lidar provides, correspondingly, high data density, precision, and convenience, we proposed a simple and efficient spatial large deflection measurement system for wind turbine blades with multi lidars. For point clouds collected from lidar scanners, registration based on point primitives and geometric primitives, dynamic radius DBSCAN clustering, spatial line clustering, and line integrals are applied to calculate the 3D coordinates of measured points on the blade. Experimentally validated, the proposed method demonstrates its effectiveness in serving as a viable alternative to the traditional pull-wire sensor measurement approach. In the minimum oscillation direction test, the measurement error is controlled within 3% compared to the theoretical value. Simultaneously, in the maximum swing direction test, the 3D coordinates of the measured point remain consistent with the changing trend observed under small deformation. These results confirm the feasibility of the system and its potentials to be generalized. Full article

Building Information Modeling (BIM) has emerged as a revolutionary tool in the domain of architectural conservation and documentation. When combined with terrestrial 3D laser scanning, it presents a powerful method to capture and represent the intricate details and nuances of historic structures. Such buildings, with their unique architectural lineage, often exude a geometric complexity unparalleled by standard designs. Consequently, the transition from scan data to a BIM framework, or the scan-to-BIM process, becomes intricate and time-intensive. Beyond the challenge of digital translation, the true essence of these historic buildings lies not only in their geometric form but also in understanding and preserving their design logic, formal composition rules, and primitive geometry. It then becomes imperative that the resulting model maintains fidelity in terms of proportion, shape, symmetry, and spatial rationale. Considering these challenges and potentials, this article delves into the process of digitalizing and BIM modeling of the Lisbon Agricultural Exhibition Pavilion located in Portugal. Our study proceeds in a tripartite structure: initiating with an in-depth terrestrial 3D laser scanning of the pavilion, followed by a comprehensive registration, processing, and alignment of the acquired scans, and culminating in a detailed BIM model using the industry-standard Revit 2020 software. Full article

This paper presents a geometric approach to solve the inverse kinematics for an anthropomorphic robotic arm with seven degrees of freedom (DoF). The proposal is based on conformal geometric algebra (CGA), by which many geometric primitives can be operated naturally and directly. CGA allows for the intersection of geometric entities such as two or more spheres or a plane’s projection over a sphere. Rigid transformations of such geometric entities are performed using only one operation through another geometric entity called a motor. CGA imposes geometric restrictions on the inverse kinematics solution, which avoids computation of the forward kinematics or other numerical solutions, unlike traditional approaches. Comparisons with state-of-the-art algorithms are included to prove our algorithm’s superior performance: such as decreased execution time and errors of the end-effector for a series of desired poses. Full article

Motivated by a ternary generalization of the Pauli exclusion principle proposed by R. Kerner, we propose a notion of a Z3-skew-symmetric covariant SO(3)-tensor of the third order, consider it as a 3-dimensional matrix, and study the geometry of the 10-dimensional complex space of these tensors. We split this 10-dimensional space into a direct sum of two 5-dimensional subspaces by means of a primitive third-order root of unity q, and in each subspace, there is an irreducible representation of the rotation group SO(3)↪SU(5). We find two SO(3)-invariants of Z3-skew-symmetric tensors: one is the canonical Hermitian metric in five-dimensional complex vector space and the other is a quadratic form denoted by K(z,z). We study the invariant properties of K(z,z) and find its stabilizer. Making use of these invariant properties, we define an SO(3)-irreducible geometric structure on a five-dimensional complex Hermitian manifold. We study a connection on a five-dimensional complex Hermitian manifold with an SO(3)-irreducible geometric structure and find its curvature and torsion. Full article

Due to the numerous objects with regular structures in indoor environments, identifying and modeling the regular objects in scenes aids indoor robots in sensing unknown environments. Typically, point cloud preprocessing can obtain highly complete object segmentation results in scenes which can be utilized as the objects for geometric analysis and modeling, thus ensuring modeling accuracy and speed. However, due to the lack of a complete object model, it is not possible to recognize and model segmented objects through matching methods. To achieve a greater understanding of scene point clouds, this paper proposes a direct geometric modeling algorithm based on segmentation results, which focuses on extracting regular geometries in the scene, rather than objects with geometric details or combinations of multiple primitives. This paper suggests using simpler geometric models to describe the corresponding point cloud data. By fully utilizing the surface structure information of segmented objects, the paper analyzes the types of faces and their relationships to classify regular geometric objects into two categories: planar and curved. Different types of geometric objects are fitted using random sampling consistency algorithms with type classification results as prior knowledge, and segmented results are modeled through a combination of size information associated with directed bounding boxes. For indoor scenes with occlusion and stacking, utilizing a higher-level semantic expression can effectively simplify the scene, complete scene abstraction and structural modeling, and aid indoor robots’ understanding and further operation in unknown environments. Full article