CN109035329A - Camera Attitude estimation optimization method based on depth characteristic - Google Patents

Camera Attitude estimation optimization method based on depth characteristic Download PDF

Info

Publication number
CN109035329A
CN109035329A CN201810878967.8A CN201810878967A CN109035329A CN 109035329 A CN109035329 A CN 109035329A CN 201810878967 A CN201810878967 A CN 201810878967A CN 109035329 A CN109035329 A CN 109035329A
Authority
CN
China
Prior art keywords
point
pixel
optimization
algorithm
function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810878967.8A
Other languages
Chinese (zh)
Inventor
纪荣嵘
郭锋
陈晗
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen University
Original Assignee
Xiamen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen University filed Critical Xiamen University
Priority to CN201810878967.8A priority Critical patent/CN109035329A/en
Publication of CN109035329A publication Critical patent/CN109035329A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/90Determination of colour characteristics

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

Camera Attitude estimation optimization method based on depth characteristic, is related to the optimization method based on supervised learning SLAM system.Using the matching algorithm based on random forest, the similitude of 2D-3D point is calculated quickly to map 2D-3D point information;Camera posture is assessed using the method for constraint function and multiple features fusion;For instability problem present in the SLAM algorithm based on deep learning, a kind of multiple features boundling optimization algorithm is proposed.Use three-dimensional reconstruction data as reference, then mapped using the related keyword point of visible 3D point and the off-line data collection from random forest, and measures posture assessment score using multiple features fusion and constraint function.Above method is for optimizing the performance based on deep learning SLAM.The results show, algorithm robustness.

Description

Camera Attitude estimation optimization method based on depth characteristic
Technical field
The present invention relates to the optimization methods based on supervised learning SLAM system, are especially for supervised learning The camera Attitude estimation optimization method based on depth characteristic of SLAM algorithm.
Background technique
SLAM technology in robot, automatic Pilot, virtually with augmented reality field have good application prospect, numerous Computer vision and artificial intelligence technology in, the research of SLAM continues intimately.In recent years, more and more robots appeared in In the visual field of people, to people life bring many conveniences, by itself camera, gyroscope, laser sensor etc. come The environment of concrete scene is obtained, and itself is positioned, particular task is completed under conditions of meeting real-time.In recent years, Domestic and international more companies put into a large amount of human and material resources to carry out the research and development of automatic driving vehicle.Unpiloted core technology SLAM technology, robust and quickly Context awareness and semantic segmentation are the unpiloted key points.It is led in augmented reality Domain, the AR application put into business scenario in the market at present be mostly based on specific template, from template recognize template with Track matching carries out actual situation interaction in conjunction with three-dimensional registration and model rendering.And really augmented reality is needed to application scenarios institute The environment at place carries out identification and semantic understanding, still needs to SLAM technology at this time as core technology.
Camera relocates a vital task in always SLAM.Based on the re-positioning method of image in SLAM It is a powerful and effective thread, the common technology of the camera Attitude estimation based on imaging is image retrieval and based on 3D scene The method of reconstruction, still, image retrieval mistake are greater than GPS location sensor.In addition, estimated accuracy also depends on data set, AR Zamir et al. ([1] Zamir, A.R., Shah, M.:Image geo-localization based on multiple nearest neighbor feature matching using generalized graphs.IEEE Trans.Pattern Anal.Mach.Intell. (2014)) the geo-location characteristic matching frame that proposes multiple arest neighbors, but if query image (Query) it mismatches so the method with data set (Database) and is severely limited.This method is better in order to realize Positioning result needs three-dimensional prior information to provide valuable spatial relationship.But the existing image based on three-dimensional priori is fixed Position method only focuses on local positioning accuracy, and ignores subsequent optimization.For local optimum, such as FastSLAM ([2] Parsons,S.:Probabilistic robotics by Sebastian Thrun,Wolfram Burgard and Dieter Fox.Knowledge Eng.Review(2006).https://doi.org/10.1017/ S0269888906210993), by sensor estimation noise to measure estimation attitude error, and it is optimized.ORB-SLAM ([3]Mur-Artal,R.,Montiel,J.M.M.,Tard′os,J.D.:ORB-SLAM:a versatile and Accurate monocular SLAM System.IEEE Trans.Robot. (2015)) it is a kind of excellent visual SLAM system System, using orb feature and part and global optimization, but it does not use depth characteristic and priori knowledge.([4]Kendall, A.,Cipolla,R.:Modelling uncertainty in deep learning for camera relocalization.In:IEEE International Conference on Robotics and Automation, ICRA 2016,Stockholm,Sweden,16–21May 2016.https://doi.org/10.1109/ ICRA.2016.7487679 a vision relocation system using Bayes) is proposed, and is returned with convolutional neural networks Camera posture has reused three-dimensional information and near real-time performance.It is proposed that method do not utilize any sensor, but be based on The similarity location model of priori three-dimensional point, still, currently based on the method for supervised learning all without quite applicable optimization Method.Although some vision positioning algorithms based on deep convolutional neural networks are considered as the end-to-end positioning side for tolerating big baseline Method still estimates the larger situation of mean error often occur when posture.Everything is attributed to no rear end optimization (such as office Portion's boundling optimization), based on the above issues, illustrate how using random forest (Cutler, A., Cutler, D.R., Stevens, J.R.:Random forests.In:Machine Learning (2012)) be 2D-3D point matching, Yi Jiru What improves and optimizes the SLAM system based on deep learning using constraint function.
Summary of the invention
The purpose of the present invention is to provide the camera Attitude estimation optimization methods based on depth characteristic.
The present invention the following steps are included:
1) matching algorithm based on random forest is used, calculates the similitude of 2D-3D point quickly to map 2D-3D point letter Breath;
2) camera posture is assessed using the method for constraint function and multiple features fusion;
3) for instability problem present in the SLAM algorithm based on deep learning, propose that a kind of multiple features boundling is excellent Change algorithm.
In step 1), matching algorithm of the use based on random forest calculates the similitude of 2D-3D point quickly to reflect The specific method for penetrating 2D-3D point information can are as follows:
Each decision tree is made of internal node and leaf node, and the prediction of decision tree can calculate between 2D pixel Similarity, then the similarity to three-dimensional space is calculated by the similarity between 2D pixel, until leaf since root node, Make to train convergence and modifying decision function repeatedly, decision function is expressed as follows:
split(p;δn)=[fn(p) > δn]
Wherein, n indicates the index of decision tree interior joint, and p is the nonleaf node for representing 2D key point, and [] is 0~1 index Function, δ are decision-making values, and f () is decision function:
F (p)=a1Dshape(p1,p2)+a2Dtexture(p1,p2)+a3Dcolor(p1,p2)
A and D () are defined in Fusion Features, if split (p;δ n) evaluate as 0, then the left side is arrived for training path branches Subtree, is otherwise branched off into the right, and p1 and p2 are pixels around key point to point, during training, three-dimensional reconstruction data Mapping comprising relevant corresponding 3D point and 2D point;Priori is divided into training data and verify data;Instruction is shown in algorithm 1 Practice frame;Objective function Q is for making training data and verify data study trend having the same, wherein Θ is verify data Into to the quantity in related training data difference path, PverificationIt is validation data set, λ is the similar of same branches of trading off The multifarious parameter of degree and different branches;Objective function is the ratio based on the verify data and training data for falling into same branches Example goes to measure similitude between points:
2D-3D point mapping training process of the algorithm 1 based on random forest is as shown in table 1:
Table 1
In step 2), the specific method using the method for constraint function and multiple features fusion assessment camera posture can Are as follows:
Assuming that Attitude estimation is the result is that camera external parameterBy x, y, z (location information) and w, p, q, r (four Element) conversion obtain, it is assumed that internal reference matrix K by EXIF label obtain (assuming that no radial deformation);Then, in conjunction with internal reference:
Transformation matrix can be obtained by image coordinate and world coordinates:
2D point and 3D point cloud are associated by this transformation matrix;Such as extract the evaluated assessment for crossing camera matrix The FALoG characteristic point of image is as inquiry data set.For query characteristics point m (x, y), can be found by above-mentioned mapping algorithm The closest feature of query characteristics searches for established nearest data set features, and random forest is by testing each decision section Query characteristics in point, finally inquiry reaches leaf node;The final mappings characteristics m' of random forest is node maximum probability in m point Corresponding points, corresponding points m' has its relevant 3D point.For each 3D point, at least one corresponding image of database, each Point can project to relevant image, in conjunction with pixel difference as error function, and use the error function iteration optimization appearance State matrix, it is believed that the error of different mappings point can pass through the pixel scale based on color, based on the sum of shape based on texture Error indicates, the scoring of the posture of the deviation ratio based on features described above the most is assessed for each characteristic use;
Related feature is as follows:
(1) based on the pixel characteristic of color.
Color characteristic can indicate the surface properties of objects in images, using the fusion picture of fusion single pixel and image-region Plain characterization method can farthest express the color characteristic of single pixel;Two are distinguished using color distance function individually Pixel color variation:
Wherein, p (x, y) and p'(x, y) it is two target points, R (), G () and B () they are respective corresponding R, G and B value; For region color feature, it is assumed that include target point in a region, it is assumed that there are 5 × 5 pixels, it is logical for each RGB Road, the center in region are target points;Extract target point (x, y) both horizontally and vertically on gradient value G (x, y):
Wherein, dhIt is the average value of two pixels of average value and right side of the difference target point between two pixels in left side; The other values in the horizontal and vertical region of central value 48 are dv, pass through dhWhat same mode was calculated.For remaining four 2 × 2 block of pixels, average down-sampling, finally, the pixel difference opposite sex based on color is:
Wherein, { 0,1 } δ ∈, if the difference of the down-sampled values of two blocks is lower than dpoint(p, p'), then δ is set as 0, otherwise for 1;dregion(p, p') is used and dpoint(p, p') identical calculation method, dG(p, p') is the difference of gradient value, η be canonical because Son;
(2) based on shape and based on the pixel characteristic of texture
As 32 × 32 pieces of center, Shape context feature is extracted, and calculates the Shape context of 25 sampled points, point Space logarithmic coordinates with block are divided into 12 × 3=36 part, and expression shape feature difference indicates transformation two using binarization Dot shape diversity,Wherein, n=36, item ∈ { 0,1 }, item=1 mean that the train value of i block is greater than average value, For texture, the central point for the circle that a radius is R is divided into eight equal angular regions;Calculate the mean chart in each region As intensity, if average value is greater than center pixel value, the value in the region is set as 1, otherwise, is set as 0;Eight binary system sequences Column, which are converted, indicates textural characteristics with decimal number;Finally, DtextureIt is determined by Hamming distances;
Finally, valuation functions E is the combination of features described above:
E(pi,pj)=a1Dshape(pi,pj)+a2Dtexture(pi,pj)+a3||Dcolor(pi,pj)||2
Wherein, ai∈ (0,1) indicates the weight of different item, DcolorIt is considered as in this fusion diversity function just Then item, DshapeAnd DtextureIt is connected, by k repetitive exercise the parameter in this function is assessed most Good, valuation functions, which describe posture estimation error, can use inquiry two dimensional character and weight of the priori data collection feature in characteristic point Point tolerance is projected to measure.
It is described for instability problem present in the SLAM algorithm based on deep learning in step 3), propose one The specific method is as follows for kind multiple features boundling optimization algorithm:
2D-3D mapping and key are stored by opening a new thread based on the deep learning SLAM system of priori knowledge Frame selects those satisfactionsKey frame as boundling optimization (bundle adjustment, abbreviation BA) pass Key frame set;Local BA optimizes all points that can be seen by key frame;Observation point facilitates constraint function and final posture Optimization, Global B A optimization is similar with part BA optimization, and only system will execute part BA between 30 frames, hold between 300 frames Row Global B A;Using after the SLAM system of optimization algorithm compared with other advanced SLAM systems it is as shown in table 2.
Table 2
The present invention combines the SLAM technology based on deep learning, the two-dimensional image point based on random forest and three-dimensional point cloud Mapping devises one based on deep learning SLAM system optimization algorithm, has well solved deep learning SLAM system optimization The blank of algorithm.SLAM environment construction, image 2D point and the 3D point cloud of the low calculation amount of system collection are matched with one, at the end PC and Mobile terminal can carry out real-time reconstruction and optimization to scene, maintain relatively high reconstruction precision, and to robot, nobody drives It sails, the fields such as augmented reality suffer from important practical value and meaning.
Most of existing localization methods are based only upon the approximate posture confidence level of reference point distance.It is different from other methods, The present invention uses three-dimensional reconstruction data as reference, then using visible 3D point and the off-line data collection from random forest The mapping of related keyword point, and posture assessment score is measured using multiple features fusion and constraint function.Above method is for optimizing Performance based on deep learning SLAM.The results show, algorithm robustness of the invention.
Detailed description of the invention
Fig. 1 is the matched system survey figure of 2D-3D point.
Fig. 2 is the diagram of 5 × 5 pixel regions based on color characteristic.
Fig. 3 is the diagram of the feature based on shape.
Specific embodiment
Following embodiment will the present invention is further illustrated in conjunction with attached drawing.
One, basic conception
1) the 2D-3D point mapping based on random forest
The purpose of construction random forest is to find the relationship between query image and three-dimensional reconstruction priori data.It can lead to Cross the posture that the mapping framework assesses different location algorithms.Since random forests algorithm has compared with low-complexity and higher robust The advantages of, so that mapping algorithm meets effective assessment in real time.Substantially, decision tree be one from 2D to 3D retrieve triangle it is several The mapping tool of what relationship, can be substituted with other methods.
The training of decision tree is the key that mapping algorithm.The performance of random forest is by the integration of each different decision tree Performance determines.Each decision tree is made of internal node and leaf node.The prediction of decision tree can calculate 2D pixel Between similarity, then by between 2D pixel similarity calculate to three-dimensional space similarity.Since root node always To leaf, make to train convergence and modifying decision function repeatedly, decision function is expressed as follows:
split(p;δn)=[fn(p) > δn]
Wherein, n indicates the index of decision tree interior joint, and p is the nonleaf node for representing 2D key point, and [] is 0-1 index letter Number, δ is decision-making value, and f () is decision function:
F (p)=a1Dshape(p1,p2)+a2Dtexture(p1,p2)+a3Dcolor(p1,p2)
A and D () are defined in Fusion Features.If split (p;δ n) evaluate as 0, then the left side is arrived for training path branches Otherwise subtree is branched off into the right.P1 and p2 is pixel around key point to point.During training, three-dimensional reconstruction data (priori data) includes the mapping of relevant corresponding 3D point and 2D point.Priori is divided into training data and verify data.It is calculating Trained frame is illustrated in method 1.It should be noted that used quick FALoG characteristic point (Wang, Z., Fan, B., Wu, F.: FRIF:fast robust invariant feature.In:British MachineVision Conference,BMVC It 2013, Bristol, UK, 9-13September 2013.https: //doi.org/10.5244/C.27.16) can be rapidly Detect corresponding binary feature point.Objective function Q is for making the study as having the same as possible of training data and verify data Trend.Wherein Θ is that verify data enters and the quantity in related training data difference path, PverificationIt is validation data set.λ It is the trade off similarity of same branches and the multifarious parameter of different branches.Objective function is based on falling into testing for same branches Card data and the ratio of training data go to measure similitude between points.
2D-3D point mapping training process of the algorithm 1 based on random forest is as shown in table 1.
2) Feature Fusion Algorithm
Although the relationship of 2D and 3D is easy to obtain by above-mentioned mapping.Due to there is no image posture label, phase is judged The quality of machine posture is one and is difficult the measurement carried out.Design a set of camera posture assessment algorithm (point of assessment camera posture Number), this algorithm does not need to carry out artificial mark, it is only necessary to the three-dimensional reconstruction data obtained by transfer learning.It predicts Camera posture can make to calculate by any camera attitude prediction algorithm.Assuming that Attitude estimation outside camera the result is that join NumberBy x, y, z (location information) and w, p, q, r (four elements) conversion are obtained.Assuming that internal reference matrix K is by EXIF label It obtains (assuming that no radial deformation).Then, in conjunction with internal reference:
Transformation matrix can be obtained by image coordinate and world coordinates:
2D point and 3D point cloud are associated by this transformation matrix.Such as extract the evaluated assessment for crossing camera matrix The FALoG characteristic point of image is as inquiry data set.For query characteristics point m (x, y), can be found by above-mentioned mapping algorithm The closest feature of query characteristics.As shown in Figure 1, the established nearest data set features of search, random forest pass through test Query characteristics in each decision node, finally inquiry reaches leaf node.The final mappings characteristics m' of random forest is section in m point The corresponding points of point maximum probability.Corresponding points m' has its relevant 3D point.For each 3D point, at least one is corresponding for database Image, each point can project to relevant image.In conjunction with pixel difference as error function, and use this error function Iteration optimization attitude matrix.Think that the error of different mappings point can be based on shape and be based on texture by based on color Pixel scale error indicate.For each feature, we are explained later.Finally, being based on features described above using assessment Deviation ratio posture the most scoring.
1. the pixel characteristic based on color.
Color characteristic can illustrate the surface properties of objects in images.In the present invention, using fusion single pixel and figure As the fusion pixel characteristic method in region can farthest express the color characteristic of single pixel.Use color distance function To distinguish two single pixel color changes:
Wherein p (x, y) and p'(x, y) it is two target points, R (), G () and B () they are respective corresponding R, G and B value.It is right In region color feature, it is assumed that include target point in a region, it is assumed that have 5 × 5 pixels, as shown in Fig. 2, for every A RGB channel, the center in region are target points.Extract target point (x, y) both horizontally and vertically on gradient value G (x, y)
Wherein dhIt is the average value of two pixels of average value and right side of the difference target point between two pixels in left side.? The other values in the horizontal and vertical region of Fig. 2 central value 48 are dv, it is by d in Fig. 2hWhat same mode was calculated.For it Remaining four 2 × 2 block of pixels (point in Fig. 2 in addition to central point vertically and horizontally region), averagely their down-sampling.Most Afterwards, the pixel difference opposite sex based on color is:
Wherein, { 0,1 } δ ∈, if the difference of the down-sampled values of two blocks is lower than dpoint(p, p'), then δ is set as 0, otherwise for 1。dregion(p, p') is used and dpoint(p, p') identical calculation method.dG(p, p') is the difference of gradient value, and η is canonical
The factor.
2. based on shape and based on the pixel characteristic of texture.
In order to faster, more efficiently extract the Feature Points of pixel, as 32 × 32 pieces of center, under extracting in shape Literary feature, and calculate the Shape context of 25 sampled points.As shown in figure 3, the space logarithmic coordinates of distribution block are divided into 12 × 3= 36 parts.Histogram is for indicating shape eigenvectors.In order to express shape feature difference, indicate to become using binarization The diversity of two o'clock shape histogram is changed,Wherein n=36, item ∈ { 0,1 }, item=1 mean the column of i block Value is greater than the average value of histogram.For texture, the central point for the circle that a radius is R is divided into eight equal angular areas Domain.The average image intensity in each region is calculated, if average value is greater than center pixel value, the value in the region is set as 1, no Then, it is set as 0.Eight binary sequences, which are converted, indicates textural characteristics with decimal number.Finally, DtextureBy Hamming distances It determines.
Finally, valuation functions E is the combination of features described above:
E(pi,pj)=a1Dshape(pi,pj)+a2Dtexture(pi,pj)+a3||Dcolor(pi,pj)||2
Wherein, aiThe weight of ∈ (0,1) expression different item.DcolorIt is considered as in this fusion diversity function just Then item.DshapeAnd DtextureIt is connected.By k repetitive exercise the parameter in this function is assessed most It is good.Valuation functions, which describe posture estimation error, can use inquiry two dimensional character and weight of the priori data collection feature in characteristic point Point tolerance is projected to measure.
3) the rear end optimization based on depth S LAM
Local positioning mistake will reflect in attitude prediction and three-dimensional reconstruction between different key frames.Key frame posture is estimated The accumulation of meter mistake results in mistake of overall importance and increases, in this way, the precision of whole system will receive limitation.Based on priori knowledge Deep learning SLAM system stores 2D-3D mapping and key frame by opening a new thread.Select those satisfactionsKey frame as boundling optimization (bundle adjustment, abbreviation BA) key frame set.Local BA Optimize all points that can be seen by key frame.Observation point facilitates the optimization of constraint function and final posture.Global B A optimization Similar with local BA optimization, only system will execute part BA between 30 frames, execute Global B A between 300 frames.
Using after the SLAM system of optimization algorithm of the invention compared with other advanced SLAM systems it is as shown in table 2.

Claims (4)

1. the camera Attitude estimation optimization method based on depth characteristic, it is characterised in that the following steps are included:
1) matching algorithm based on random forest is used, calculates the similitude of 2D-3D point quickly to map 2D-3D point information;
2) camera posture is assessed using the method for constraint function and multiple features fusion;
3) for instability problem present in the SLAM algorithm based on deep learning, propose that a kind of multiple features boundling optimization is calculated Method.
2. the camera Attitude estimation optimization method based on depth characteristic as described in claim 1, it is characterised in that in step 1), Matching algorithm of the use based on random forest calculates the similitude of 2D-3D point quickly to map the specific of 2D-3D point information Method are as follows:
Each decision tree is made of internal node and leaf node, and the prediction of decision tree can calculate similar between 2D pixel Degree, then the similarity to three-dimensional space is calculated by the similarity between 2D pixel, until leaf, passes through since root node It modifies decision function repeatedly and makes to train convergence, decision function is expressed as follows:
split(p;δn)=[fn(p) > δn]
Wherein, n indicates the index of decision tree interior joint, and p is the nonleaf node for representing 2D key point, and [] is 0~1 target function, δ is decision-making value, and f () is decision function:
F (p)=a1Dshape(p1,p2)+a2Dtexture(p1,p2)+a3Dcolor(p1,p2)
A and D () are defined in Fusion Features, if split (p;δ n) evaluate as 0, then it is the subtree of training path branches to the left side, Otherwise it is branched off into the right, p1 and p2 are pixels around key point to point, and during training, three-dimensional reconstruction data include phase The mapping of corresponding the 3D point and 2D point that close;Priori is divided into training data and verify data;Training frame is shown in algorithm 1 Frame;Objective function Q is for making training data and verify data study trend having the same, wherein Θ is that verify data enters To the quantity in related training data difference path, PverificationValidation data set, λ be trade off same branches similarity and The multifarious parameter of different branches;Objective function is that the ratio based on the verify data and training data that fall into same branches is gone Measure similitude between points:
2D-3D point mapping training process of the algorithm 1 based on random forest is as shown in table 1:
Table 1
3. the camera Attitude estimation optimization method based on depth characteristic as described in claim 1, it is characterised in that in step 2), It is described to assess camera posture using the method for constraint function and multiple features fusion method particularly includes:
Assuming that Attitude estimation is the result is that camera external parameterBy x, y, z and w, p, q, r converts to obtain, it is assumed that internal reference square Battle array K is obtained by EXIF label;Then, in conjunction with internal reference:
Transformation matrix is obtained by image coordinate and world coordinates:
2D point and 3D point cloud are associated by this transformation matrix;Extract the evaluated assessment image for crossing camera matrix FALoG characteristic point is as inquiry data set;For query characteristics point m (x, y), query characteristics are found by above-mentioned mapping algorithm Closest feature, searches for established nearest data set features, and random forest passes through the inquiry in each decision node of test Feature, finally inquiry reaches leaf node;The final mappings characteristics m' of random forest is the corresponding points of node maximum probability in m point, Corresponding points m' has its relevant 3D point;For each 3D point, at least one corresponding image of database, each point projects to phase The image of pass in conjunction with pixel difference as error function, and uses the error function iteration optimization attitude matrix, it is believed that different The error of mapping point based on the sum of shape based on the pixel scale error of texture by being indicated, for each spy based on color The scoring of posture of the sign using deviation ratio of the assessment based on features described above the most;
Related feature is as follows:
(1) based on the pixel characteristic of color
Color characteristic can indicate the surface properties of objects in images, and the fusion pixel using fusion single pixel and image-region is special Sign method farthest expresses the color characteristic of single pixel;Two single pixel colors are distinguished using color distance function Variation:
Wherein, p (x, y) and p'(x, y) it is two target points, R (), G () and B () they are respective corresponding R, G and B value;For Region color feature, it is assumed that include target point in a region, it is assumed that have 5 × 5 pixels, for each RGB channel, area The center in domain is target point;Extract target point (x, y) both horizontally and vertically on gradient value G (x, y):
Wherein, dhIt is the average value of two pixels of average value and right side of the difference target point between two pixels in left side;Central value The other values in 48 horizontal and vertical regions are dv, pass through dhWhat same mode was calculated;For remaining four 2 × 2 pixel Block, average down-sampling, finally, the pixel difference opposite sex based on color is:
Wherein, { 0,1 } δ ∈, if the difference of the down-sampled values of two blocks is lower than dpoint(p, p'), then δ is set as 0, is otherwise 1; dregion(p, p') is used and dpoint(p, p') identical calculation method, dG(p, p') is the difference of gradient value, and η is regular factor;
(2) based on shape and based on the pixel characteristic of texture
As 32 × 32 pieces of center, Shape context feature is extracted, and calculates the Shape context of 25 sampled points, distributes block Space logarithmic coordinates be divided into 12 × 3=36 part, expression shape feature difference indicates two dots of transformation using binarization Shape diversity,Wherein, n=36, item ∈ { 0,1 }, item=1 mean that the train value of i block is greater than average value, right It is divided into eight equal angular regions in the central point of texture, the circle that a radius is R;Calculate the average image in each region Intensity, if average value is greater than center pixel value, the value in the region is set as 1, otherwise, is set as 0;Eight binary sequences Textural characteristics are indicated with decimal number by converting;Finally, DtextureIt is determined by Hamming distances;
Finally, valuation functions E is the combination of features described above:
E(pi,pj)=a1Dshape(pi,pj)+a2Dtexture(pi,pj)+a3||Dcolor(pi,pj)||2
Wherein, ai∈ (0,1) indicates the weight of different item, DcolorThe canonical being considered as in this fusion diversity function , DshapeAnd DtextureIt is connected, by k repetitive exercise the parameter in this function is assessed best, Valuation functions describe posture estimation error inquiry two dimensional character and re-projection point of the priori data collection feature in characteristic point Error is measured.
4. the camera Attitude estimation optimization method based on depth characteristic as described in claim 1, it is characterised in that in step 3), It is described to be directed to instability problem present in the SLAM algorithm based on deep learning, propose a kind of multiple features boundling optimization algorithm The specific method is as follows:
2D-3D mapping and key frame are stored by opening a new thread based on the deep learning SLAM system of priori knowledge, Select those satisfactionsKey frame as boundling optimization (bundle adjustment, abbreviation BA) key frame Set;Local BA optimizes all points seen by key frame;Observation point facilitates the optimization of constraint function and final posture, global BA optimization is similar with part BA optimization, and only system will execute part BA between 30 frames, executes Global B A between 300 frames;
Table 2
Using after the SLAM system of optimization algorithm compared with other advanced SLAM systems it is as shown in table 2.
CN201810878967.8A 2018-08-03 2018-08-03 Camera Attitude estimation optimization method based on depth characteristic Pending CN109035329A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810878967.8A CN109035329A (en) 2018-08-03 2018-08-03 Camera Attitude estimation optimization method based on depth characteristic

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810878967.8A CN109035329A (en) 2018-08-03 2018-08-03 Camera Attitude estimation optimization method based on depth characteristic

Publications (1)

Publication Number Publication Date
CN109035329A true CN109035329A (en) 2018-12-18

Family

ID=64648411

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810878967.8A Pending CN109035329A (en) 2018-08-03 2018-08-03 Camera Attitude estimation optimization method based on depth characteristic

Country Status (1)

Country Link
CN (1) CN109035329A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009732A (en) * 2019-04-11 2019-07-12 司岚光电科技(苏州)有限公司 Based on GMS characteristic matching towards complicated large scale scene three-dimensional reconstruction method
CN110033007A (en) * 2019-04-19 2019-07-19 福州大学 Attribute recognition approach is worn clothes based on the pedestrian of depth attitude prediction and multiple features fusion
CN112101802A (en) * 2020-09-21 2020-12-18 广东电网有限责任公司电力科学研究院 Attitude load data evaluation method and device, electronic equipment and storage medium
WO2021017314A1 (en) * 2019-07-29 2021-02-04 浙江商汤科技开发有限公司 Information processing method, information positioning method and apparatus, electronic device and storage medium
CN112733761A (en) * 2021-01-15 2021-04-30 浙江工业大学 Human body state matching method based on machine learning
CN112967296A (en) * 2021-03-10 2021-06-15 重庆理工大学 Point cloud dynamic region graph convolution method, classification method and segmentation method
CN113299073A (en) * 2021-04-28 2021-08-24 北京百度网讯科技有限公司 Method, device, equipment and storage medium for identifying illegal parking of vehicle
WO2022252118A1 (en) * 2021-06-01 2022-12-08 华为技术有限公司 Head posture measurement method and apparatus
CN116051723A (en) * 2022-08-03 2023-05-02 荣耀终端有限公司 Bundling adjustment method and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170249491A1 (en) * 2011-08-30 2017-08-31 Digimarc Corporation Methods and arrangements for identifying objects
CN107833253A (en) * 2017-09-22 2018-03-23 北京航空航天大学青岛研究院 A kind of camera pose refinement method towards the generation of RGBD three-dimensional reconstructions texture
CN108230337A (en) * 2017-12-31 2018-06-29 厦门大学 A kind of method that semantic SLAM systems based on mobile terminal are realized

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170249491A1 (en) * 2011-08-30 2017-08-31 Digimarc Corporation Methods and arrangements for identifying objects
CN107833253A (en) * 2017-09-22 2018-03-23 北京航空航天大学青岛研究院 A kind of camera pose refinement method towards the generation of RGBD three-dimensional reconstructions texture
CN108230337A (en) * 2017-12-31 2018-06-29 厦门大学 A kind of method that semantic SLAM systems based on mobile terminal are realized

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HAN CHEN等: "Optimization Algorithm Toward Deep Features Based Camera Pose Estimation", 《ICIG 2017: IMAGE AND GRAPHICS》 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009732B (en) * 2019-04-11 2023-10-03 司岚光电科技(苏州)有限公司 GMS feature matching-based three-dimensional reconstruction method for complex large-scale scene
CN110009732A (en) * 2019-04-11 2019-07-12 司岚光电科技(苏州)有限公司 Based on GMS characteristic matching towards complicated large scale scene three-dimensional reconstruction method
CN110033007A (en) * 2019-04-19 2019-07-19 福州大学 Attribute recognition approach is worn clothes based on the pedestrian of depth attitude prediction and multiple features fusion
CN110033007B (en) * 2019-04-19 2022-08-09 福州大学 Pedestrian clothing attribute identification method based on depth attitude estimation and multi-feature fusion
WO2021017314A1 (en) * 2019-07-29 2021-02-04 浙江商汤科技开发有限公司 Information processing method, information positioning method and apparatus, electronic device and storage medium
US11983820B2 (en) 2019-07-29 2024-05-14 Zhejiang Sensetime Technology Development Co., Ltd Information processing method and device, positioning method and device, electronic device and storage medium
CN112101802A (en) * 2020-09-21 2020-12-18 广东电网有限责任公司电力科学研究院 Attitude load data evaluation method and device, electronic equipment and storage medium
CN112733761A (en) * 2021-01-15 2021-04-30 浙江工业大学 Human body state matching method based on machine learning
CN112733761B (en) * 2021-01-15 2024-03-19 浙江工业大学 Human body state matching method based on machine learning
CN112967296A (en) * 2021-03-10 2021-06-15 重庆理工大学 Point cloud dynamic region graph convolution method, classification method and segmentation method
CN113299073A (en) * 2021-04-28 2021-08-24 北京百度网讯科技有限公司 Method, device, equipment and storage medium for identifying illegal parking of vehicle
US12039864B2 (en) 2021-04-28 2024-07-16 Beijing Baidu Netcom Science Technology Co., Ltd. Method of recognizing illegal parking of vehicle, device and storage medium
WO2022252118A1 (en) * 2021-06-01 2022-12-08 华为技术有限公司 Head posture measurement method and apparatus
CN116051723B (en) * 2022-08-03 2023-10-20 荣耀终端有限公司 Bundling adjustment method and electronic equipment
CN116051723A (en) * 2022-08-03 2023-05-02 荣耀终端有限公司 Bundling adjustment method and electronic equipment

Similar Documents

Publication Publication Date Title
CN109035329A (en) Camera Attitude estimation optimization method based on depth characteristic
CN111563442B (en) Slam method and system for fusing point cloud and camera image data based on laser radar
Feng et al. 2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud
Xia et al. Geometric primitives in LiDAR point clouds: A review
CN111080627B (en) 2D +3D large airplane appearance defect detection and analysis method based on deep learning
CN109410321B (en) Three-dimensional reconstruction method based on convolutional neural network
Li et al. A tutorial review on point cloud registrations: principle, classification, comparison, and technology challenges
CN113012122B (en) Category-level 6D pose and size estimation method and device
CN109559320A (en) Realize that vision SLAM semanteme builds the method and system of figure function based on empty convolution deep neural network
Yu et al. Robust robot pose estimation for challenging scenes with an RGB-D camera
CN111709988B (en) Method and device for determining characteristic information of object, electronic equipment and storage medium
CN110223351B (en) Depth camera positioning method based on convolutional neural network
CN115546116B (en) Full-coverage type rock mass discontinuous surface extraction and interval calculation method and system
Wu et al. An object slam framework for association, mapping, and high-level tasks
KR101460313B1 (en) Apparatus and method for robot localization using visual feature and geometric constraints
Dewan et al. Learning a local feature descriptor for 3d lidar scans
Rubio et al. Efficient monocular pose estimation for complex 3D models
CN104182747A (en) Object detection and tracking method and device based on multiple stereo cameras
Wietrzykowski et al. PlaneLoc: Probabilistic global localization in 3-D using local planar features
Jiang et al. Leveraging vocabulary tree for simultaneous match pair selection and guided feature matching of UAV images
Zhang et al. Improved feature point extraction method of ORB-SLAM2 dense map
CN114358133A (en) Method for detecting looped frames based on semantic-assisted binocular vision SLAM
Álvarez et al. Junction assisted 3d pose retrieval of untextured 3d models in monocular images
Hou et al. Fast 2d map matching based on area graphs
CN111882663A (en) Visual SLAM closed-loop detection method achieved by fusing semantic information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20181218