CN109035329A - Camera Attitude estimation optimization method based on depth characteristic - Google Patents
Camera Attitude estimation optimization method based on depth characteristic Download PDFInfo
- Publication number
- CN109035329A CN109035329A CN201810878967.8A CN201810878967A CN109035329A CN 109035329 A CN109035329 A CN 109035329A CN 201810878967 A CN201810878967 A CN 201810878967A CN 109035329 A CN109035329 A CN 109035329A
- Authority
- CN
- China
- Prior art keywords
- point
- pixel
- optimization
- algorithm
- function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
Camera Attitude estimation optimization method based on depth characteristic, is related to the optimization method based on supervised learning SLAM system.Using the matching algorithm based on random forest, the similitude of 2D-3D point is calculated quickly to map 2D-3D point information;Camera posture is assessed using the method for constraint function and multiple features fusion;For instability problem present in the SLAM algorithm based on deep learning, a kind of multiple features boundling optimization algorithm is proposed.Use three-dimensional reconstruction data as reference, then mapped using the related keyword point of visible 3D point and the off-line data collection from random forest, and measures posture assessment score using multiple features fusion and constraint function.Above method is for optimizing the performance based on deep learning SLAM.The results show, algorithm robustness.
Description
Technical field
The present invention relates to the optimization methods based on supervised learning SLAM system, are especially for supervised learning
The camera Attitude estimation optimization method based on depth characteristic of SLAM algorithm.
Background technique
SLAM technology in robot, automatic Pilot, virtually with augmented reality field have good application prospect, numerous
Computer vision and artificial intelligence technology in, the research of SLAM continues intimately.In recent years, more and more robots appeared in
In the visual field of people, to people life bring many conveniences, by itself camera, gyroscope, laser sensor etc. come
The environment of concrete scene is obtained, and itself is positioned, particular task is completed under conditions of meeting real-time.In recent years,
Domestic and international more companies put into a large amount of human and material resources to carry out the research and development of automatic driving vehicle.Unpiloted core technology
SLAM technology, robust and quickly Context awareness and semantic segmentation are the unpiloted key points.It is led in augmented reality
Domain, the AR application put into business scenario in the market at present be mostly based on specific template, from template recognize template with
Track matching carries out actual situation interaction in conjunction with three-dimensional registration and model rendering.And really augmented reality is needed to application scenarios institute
The environment at place carries out identification and semantic understanding, still needs to SLAM technology at this time as core technology.
Camera relocates a vital task in always SLAM.Based on the re-positioning method of image in SLAM
It is a powerful and effective thread, the common technology of the camera Attitude estimation based on imaging is image retrieval and based on 3D scene
The method of reconstruction, still, image retrieval mistake are greater than GPS location sensor.In addition, estimated accuracy also depends on data set, AR
Zamir et al. ([1] Zamir, A.R., Shah, M.:Image geo-localization based on multiple
nearest neighbor feature matching using generalized graphs.IEEE Trans.Pattern
Anal.Mach.Intell. (2014)) the geo-location characteristic matching frame that proposes multiple arest neighbors, but if query image
(Query) it mismatches so the method with data set (Database) and is severely limited.This method is better in order to realize
Positioning result needs three-dimensional prior information to provide valuable spatial relationship.But the existing image based on three-dimensional priori is fixed
Position method only focuses on local positioning accuracy, and ignores subsequent optimization.For local optimum, such as FastSLAM ([2]
Parsons,S.:Probabilistic robotics by Sebastian Thrun,Wolfram Burgard and
Dieter Fox.Knowledge Eng.Review(2006).https://doi.org/10.1017/
S0269888906210993), by sensor estimation noise to measure estimation attitude error, and it is optimized.ORB-SLAM
([3]Mur-Artal,R.,Montiel,J.M.M.,Tard′os,J.D.:ORB-SLAM:a versatile and
Accurate monocular SLAM System.IEEE Trans.Robot. (2015)) it is a kind of excellent visual SLAM system
System, using orb feature and part and global optimization, but it does not use depth characteristic and priori knowledge.([4]Kendall,
A.,Cipolla,R.:Modelling uncertainty in deep learning for camera
relocalization.In:IEEE International Conference on Robotics and Automation,
ICRA 2016,Stockholm,Sweden,16–21May 2016.https://doi.org/10.1109/
ICRA.2016.7487679 a vision relocation system using Bayes) is proposed, and is returned with convolutional neural networks
Camera posture has reused three-dimensional information and near real-time performance.It is proposed that method do not utilize any sensor, but be based on
The similarity location model of priori three-dimensional point, still, currently based on the method for supervised learning all without quite applicable optimization
Method.Although some vision positioning algorithms based on deep convolutional neural networks are considered as the end-to-end positioning side for tolerating big baseline
Method still estimates the larger situation of mean error often occur when posture.Everything is attributed to no rear end optimization (such as office
Portion's boundling optimization), based on the above issues, illustrate how using random forest (Cutler, A., Cutler, D.R.,
Stevens, J.R.:Random forests.In:Machine Learning (2012)) be 2D-3D point matching, Yi Jiru
What improves and optimizes the SLAM system based on deep learning using constraint function.
Summary of the invention
The purpose of the present invention is to provide the camera Attitude estimation optimization methods based on depth characteristic.
The present invention the following steps are included:
1) matching algorithm based on random forest is used, calculates the similitude of 2D-3D point quickly to map 2D-3D point letter
Breath;
2) camera posture is assessed using the method for constraint function and multiple features fusion;
3) for instability problem present in the SLAM algorithm based on deep learning, propose that a kind of multiple features boundling is excellent
Change algorithm.
In step 1), matching algorithm of the use based on random forest calculates the similitude of 2D-3D point quickly to reflect
The specific method for penetrating 2D-3D point information can are as follows:
Each decision tree is made of internal node and leaf node, and the prediction of decision tree can calculate between 2D pixel
Similarity, then the similarity to three-dimensional space is calculated by the similarity between 2D pixel, until leaf since root node,
Make to train convergence and modifying decision function repeatedly, decision function is expressed as follows:
split(p;δn)=[fn(p) > δn]
Wherein, n indicates the index of decision tree interior joint, and p is the nonleaf node for representing 2D key point, and [] is 0~1 index
Function, δ are decision-making values, and f () is decision function:
F (p)=a1Dshape(p1,p2)+a2Dtexture(p1,p2)+a3Dcolor(p1,p2)
A and D () are defined in Fusion Features, if split (p;δ n) evaluate as 0, then the left side is arrived for training path branches
Subtree, is otherwise branched off into the right, and p1 and p2 are pixels around key point to point, during training, three-dimensional reconstruction data
Mapping comprising relevant corresponding 3D point and 2D point;Priori is divided into training data and verify data;Instruction is shown in algorithm 1
Practice frame;Objective function Q is for making training data and verify data study trend having the same, wherein Θ is verify data
Into to the quantity in related training data difference path, PverificationIt is validation data set, λ is the similar of same branches of trading off
The multifarious parameter of degree and different branches;Objective function is the ratio based on the verify data and training data for falling into same branches
Example goes to measure similitude between points:
2D-3D point mapping training process of the algorithm 1 based on random forest is as shown in table 1:
Table 1
In step 2), the specific method using the method for constraint function and multiple features fusion assessment camera posture can
Are as follows:
Assuming that Attitude estimation is the result is that camera external parameterBy x, y, z (location information) and w, p, q, r (four
Element) conversion obtain, it is assumed that internal reference matrix K by EXIF label obtain (assuming that no radial deformation);Then, in conjunction with internal reference:
Transformation matrix can be obtained by image coordinate and world coordinates:
2D point and 3D point cloud are associated by this transformation matrix;Such as extract the evaluated assessment for crossing camera matrix
The FALoG characteristic point of image is as inquiry data set.For query characteristics point m (x, y), can be found by above-mentioned mapping algorithm
The closest feature of query characteristics searches for established nearest data set features, and random forest is by testing each decision section
Query characteristics in point, finally inquiry reaches leaf node;The final mappings characteristics m' of random forest is node maximum probability in m point
Corresponding points, corresponding points m' has its relevant 3D point.For each 3D point, at least one corresponding image of database, each
Point can project to relevant image, in conjunction with pixel difference as error function, and use the error function iteration optimization appearance
State matrix, it is believed that the error of different mappings point can pass through the pixel scale based on color, based on the sum of shape based on texture
Error indicates, the scoring of the posture of the deviation ratio based on features described above the most is assessed for each characteristic use;
Related feature is as follows:
(1) based on the pixel characteristic of color.
Color characteristic can indicate the surface properties of objects in images, using the fusion picture of fusion single pixel and image-region
Plain characterization method can farthest express the color characteristic of single pixel;Two are distinguished using color distance function individually
Pixel color variation:
Wherein, p (x, y) and p'(x, y) it is two target points, R (), G () and B () they are respective corresponding R, G and B value;
For region color feature, it is assumed that include target point in a region, it is assumed that there are 5 × 5 pixels, it is logical for each RGB
Road, the center in region are target points;Extract target point (x, y) both horizontally and vertically on gradient value G (x, y):
Wherein, dhIt is the average value of two pixels of average value and right side of the difference target point between two pixels in left side;
The other values in the horizontal and vertical region of central value 48 are dv, pass through dhWhat same mode was calculated.For remaining four 2
× 2 block of pixels, average down-sampling, finally, the pixel difference opposite sex based on color is:
Wherein, { 0,1 } δ ∈, if the difference of the down-sampled values of two blocks is lower than dpoint(p, p'), then δ is set as 0, otherwise for
1;dregion(p, p') is used and dpoint(p, p') identical calculation method, dG(p, p') is the difference of gradient value, η be canonical because
Son;
(2) based on shape and based on the pixel characteristic of texture
As 32 × 32 pieces of center, Shape context feature is extracted, and calculates the Shape context of 25 sampled points, point
Space logarithmic coordinates with block are divided into 12 × 3=36 part, and expression shape feature difference indicates transformation two using binarization
Dot shape diversity,Wherein, n=36, item ∈ { 0,1 }, item=1 mean that the train value of i block is greater than average value,
For texture, the central point for the circle that a radius is R is divided into eight equal angular regions;Calculate the mean chart in each region
As intensity, if average value is greater than center pixel value, the value in the region is set as 1, otherwise, is set as 0;Eight binary system sequences
Column, which are converted, indicates textural characteristics with decimal number;Finally, DtextureIt is determined by Hamming distances;
Finally, valuation functions E is the combination of features described above:
E(pi,pj)=a1Dshape(pi,pj)+a2Dtexture(pi,pj)+a3||Dcolor(pi,pj)||2
Wherein, ai∈ (0,1) indicates the weight of different item, DcolorIt is considered as in this fusion diversity function just
Then item, DshapeAnd DtextureIt is connected, by k repetitive exercise the parameter in this function is assessed most
Good, valuation functions, which describe posture estimation error, can use inquiry two dimensional character and weight of the priori data collection feature in characteristic point
Point tolerance is projected to measure.
It is described for instability problem present in the SLAM algorithm based on deep learning in step 3), propose one
The specific method is as follows for kind multiple features boundling optimization algorithm:
2D-3D mapping and key are stored by opening a new thread based on the deep learning SLAM system of priori knowledge
Frame selects those satisfactionsKey frame as boundling optimization (bundle adjustment, abbreviation BA) pass
Key frame set;Local BA optimizes all points that can be seen by key frame;Observation point facilitates constraint function and final posture
Optimization, Global B A optimization is similar with part BA optimization, and only system will execute part BA between 30 frames, hold between 300 frames
Row Global B A;Using after the SLAM system of optimization algorithm compared with other advanced SLAM systems it is as shown in table 2.
Table 2
The present invention combines the SLAM technology based on deep learning, the two-dimensional image point based on random forest and three-dimensional point cloud
Mapping devises one based on deep learning SLAM system optimization algorithm, has well solved deep learning SLAM system optimization
The blank of algorithm.SLAM environment construction, image 2D point and the 3D point cloud of the low calculation amount of system collection are matched with one, at the end PC and
Mobile terminal can carry out real-time reconstruction and optimization to scene, maintain relatively high reconstruction precision, and to robot, nobody drives
It sails, the fields such as augmented reality suffer from important practical value and meaning.
Most of existing localization methods are based only upon the approximate posture confidence level of reference point distance.It is different from other methods,
The present invention uses three-dimensional reconstruction data as reference, then using visible 3D point and the off-line data collection from random forest
The mapping of related keyword point, and posture assessment score is measured using multiple features fusion and constraint function.Above method is for optimizing
Performance based on deep learning SLAM.The results show, algorithm robustness of the invention.
Detailed description of the invention
Fig. 1 is the matched system survey figure of 2D-3D point.
Fig. 2 is the diagram of 5 × 5 pixel regions based on color characteristic.
Fig. 3 is the diagram of the feature based on shape.
Specific embodiment
Following embodiment will the present invention is further illustrated in conjunction with attached drawing.
One, basic conception
1) the 2D-3D point mapping based on random forest
The purpose of construction random forest is to find the relationship between query image and three-dimensional reconstruction priori data.It can lead to
Cross the posture that the mapping framework assesses different location algorithms.Since random forests algorithm has compared with low-complexity and higher robust
The advantages of, so that mapping algorithm meets effective assessment in real time.Substantially, decision tree be one from 2D to 3D retrieve triangle it is several
The mapping tool of what relationship, can be substituted with other methods.
The training of decision tree is the key that mapping algorithm.The performance of random forest is by the integration of each different decision tree
Performance determines.Each decision tree is made of internal node and leaf node.The prediction of decision tree can calculate 2D pixel
Between similarity, then by between 2D pixel similarity calculate to three-dimensional space similarity.Since root node always
To leaf, make to train convergence and modifying decision function repeatedly, decision function is expressed as follows:
split(p;δn)=[fn(p) > δn]
Wherein, n indicates the index of decision tree interior joint, and p is the nonleaf node for representing 2D key point, and [] is 0-1 index letter
Number, δ is decision-making value, and f () is decision function:
F (p)=a1Dshape(p1,p2)+a2Dtexture(p1,p2)+a3Dcolor(p1,p2)
A and D () are defined in Fusion Features.If split (p;δ n) evaluate as 0, then the left side is arrived for training path branches
Otherwise subtree is branched off into the right.P1 and p2 is pixel around key point to point.During training, three-dimensional reconstruction data
(priori data) includes the mapping of relevant corresponding 3D point and 2D point.Priori is divided into training data and verify data.It is calculating
Trained frame is illustrated in method 1.It should be noted that used quick FALoG characteristic point (Wang, Z., Fan, B., Wu, F.:
FRIF:fast robust invariant feature.In:British MachineVision Conference,BMVC
It 2013, Bristol, UK, 9-13September 2013.https: //doi.org/10.5244/C.27.16) can be rapidly
Detect corresponding binary feature point.Objective function Q is for making the study as having the same as possible of training data and verify data
Trend.Wherein Θ is that verify data enters and the quantity in related training data difference path, PverificationIt is validation data set.λ
It is the trade off similarity of same branches and the multifarious parameter of different branches.Objective function is based on falling into testing for same branches
Card data and the ratio of training data go to measure similitude between points.
2D-3D point mapping training process of the algorithm 1 based on random forest is as shown in table 1.
2) Feature Fusion Algorithm
Although the relationship of 2D and 3D is easy to obtain by above-mentioned mapping.Due to there is no image posture label, phase is judged
The quality of machine posture is one and is difficult the measurement carried out.Design a set of camera posture assessment algorithm (point of assessment camera posture
Number), this algorithm does not need to carry out artificial mark, it is only necessary to the three-dimensional reconstruction data obtained by transfer learning.It predicts
Camera posture can make to calculate by any camera attitude prediction algorithm.Assuming that Attitude estimation outside camera the result is that join
NumberBy x, y, z (location information) and w, p, q, r (four elements) conversion are obtained.Assuming that internal reference matrix K is by EXIF label
It obtains (assuming that no radial deformation).Then, in conjunction with internal reference:
Transformation matrix can be obtained by image coordinate and world coordinates:
2D point and 3D point cloud are associated by this transformation matrix.Such as extract the evaluated assessment for crossing camera matrix
The FALoG characteristic point of image is as inquiry data set.For query characteristics point m (x, y), can be found by above-mentioned mapping algorithm
The closest feature of query characteristics.As shown in Figure 1, the established nearest data set features of search, random forest pass through test
Query characteristics in each decision node, finally inquiry reaches leaf node.The final mappings characteristics m' of random forest is section in m point
The corresponding points of point maximum probability.Corresponding points m' has its relevant 3D point.For each 3D point, at least one is corresponding for database
Image, each point can project to relevant image.In conjunction with pixel difference as error function, and use this error function
Iteration optimization attitude matrix.Think that the error of different mappings point can be based on shape and be based on texture by based on color
Pixel scale error indicate.For each feature, we are explained later.Finally, being based on features described above using assessment
Deviation ratio posture the most scoring.
1. the pixel characteristic based on color.
Color characteristic can illustrate the surface properties of objects in images.In the present invention, using fusion single pixel and figure
As the fusion pixel characteristic method in region can farthest express the color characteristic of single pixel.Use color distance function
To distinguish two single pixel color changes:
Wherein p (x, y) and p'(x, y) it is two target points, R (), G () and B () they are respective corresponding R, G and B value.It is right
In region color feature, it is assumed that include target point in a region, it is assumed that have 5 × 5 pixels, as shown in Fig. 2, for every
A RGB channel, the center in region are target points.Extract target point (x, y) both horizontally and vertically on gradient value G (x, y)
Wherein dhIt is the average value of two pixels of average value and right side of the difference target point between two pixels in left side.?
The other values in the horizontal and vertical region of Fig. 2 central value 48 are dv, it is by d in Fig. 2hWhat same mode was calculated.For it
Remaining four 2 × 2 block of pixels (point in Fig. 2 in addition to central point vertically and horizontally region), averagely their down-sampling.Most
Afterwards, the pixel difference opposite sex based on color is:
Wherein, { 0,1 } δ ∈, if the difference of the down-sampled values of two blocks is lower than dpoint(p, p'), then δ is set as 0, otherwise for
1。dregion(p, p') is used and dpoint(p, p') identical calculation method.dG(p, p') is the difference of gradient value, and η is canonical
The factor.
2. based on shape and based on the pixel characteristic of texture.
In order to faster, more efficiently extract the Feature Points of pixel, as 32 × 32 pieces of center, under extracting in shape
Literary feature, and calculate the Shape context of 25 sampled points.As shown in figure 3, the space logarithmic coordinates of distribution block are divided into 12 × 3=
36 parts.Histogram is for indicating shape eigenvectors.In order to express shape feature difference, indicate to become using binarization
The diversity of two o'clock shape histogram is changed,Wherein n=36, item ∈ { 0,1 }, item=1 mean the column of i block
Value is greater than the average value of histogram.For texture, the central point for the circle that a radius is R is divided into eight equal angular areas
Domain.The average image intensity in each region is calculated, if average value is greater than center pixel value, the value in the region is set as 1, no
Then, it is set as 0.Eight binary sequences, which are converted, indicates textural characteristics with decimal number.Finally, DtextureBy Hamming distances
It determines.
Finally, valuation functions E is the combination of features described above:
E(pi,pj)=a1Dshape(pi,pj)+a2Dtexture(pi,pj)+a3||Dcolor(pi,pj)||2
Wherein, aiThe weight of ∈ (0,1) expression different item.DcolorIt is considered as in this fusion diversity function just
Then item.DshapeAnd DtextureIt is connected.By k repetitive exercise the parameter in this function is assessed most
It is good.Valuation functions, which describe posture estimation error, can use inquiry two dimensional character and weight of the priori data collection feature in characteristic point
Point tolerance is projected to measure.
3) the rear end optimization based on depth S LAM
Local positioning mistake will reflect in attitude prediction and three-dimensional reconstruction between different key frames.Key frame posture is estimated
The accumulation of meter mistake results in mistake of overall importance and increases, in this way, the precision of whole system will receive limitation.Based on priori knowledge
Deep learning SLAM system stores 2D-3D mapping and key frame by opening a new thread.Select those satisfactionsKey frame as boundling optimization (bundle adjustment, abbreviation BA) key frame set.Local BA
Optimize all points that can be seen by key frame.Observation point facilitates the optimization of constraint function and final posture.Global B A optimization
Similar with local BA optimization, only system will execute part BA between 30 frames, execute Global B A between 300 frames.
Using after the SLAM system of optimization algorithm of the invention compared with other advanced SLAM systems it is as shown in table 2.
Claims (4)
1. the camera Attitude estimation optimization method based on depth characteristic, it is characterised in that the following steps are included:
1) matching algorithm based on random forest is used, calculates the similitude of 2D-3D point quickly to map 2D-3D point information;
2) camera posture is assessed using the method for constraint function and multiple features fusion;
3) for instability problem present in the SLAM algorithm based on deep learning, propose that a kind of multiple features boundling optimization is calculated
Method.
2. the camera Attitude estimation optimization method based on depth characteristic as described in claim 1, it is characterised in that in step 1),
Matching algorithm of the use based on random forest calculates the similitude of 2D-3D point quickly to map the specific of 2D-3D point information
Method are as follows:
Each decision tree is made of internal node and leaf node, and the prediction of decision tree can calculate similar between 2D pixel
Degree, then the similarity to three-dimensional space is calculated by the similarity between 2D pixel, until leaf, passes through since root node
It modifies decision function repeatedly and makes to train convergence, decision function is expressed as follows:
split(p;δn)=[fn(p) > δn]
Wherein, n indicates the index of decision tree interior joint, and p is the nonleaf node for representing 2D key point, and [] is 0~1 target function,
δ is decision-making value, and f () is decision function:
F (p)=a1Dshape(p1,p2)+a2Dtexture(p1,p2)+a3Dcolor(p1,p2)
A and D () are defined in Fusion Features, if split (p;δ n) evaluate as 0, then it is the subtree of training path branches to the left side,
Otherwise it is branched off into the right, p1 and p2 are pixels around key point to point, and during training, three-dimensional reconstruction data include phase
The mapping of corresponding the 3D point and 2D point that close;Priori is divided into training data and verify data;Training frame is shown in algorithm 1
Frame;Objective function Q is for making training data and verify data study trend having the same, wherein Θ is that verify data enters
To the quantity in related training data difference path, PverificationValidation data set, λ be trade off same branches similarity and
The multifarious parameter of different branches;Objective function is that the ratio based on the verify data and training data that fall into same branches is gone
Measure similitude between points:
2D-3D point mapping training process of the algorithm 1 based on random forest is as shown in table 1:
Table 1
3. the camera Attitude estimation optimization method based on depth characteristic as described in claim 1, it is characterised in that in step 2),
It is described to assess camera posture using the method for constraint function and multiple features fusion method particularly includes:
Assuming that Attitude estimation is the result is that camera external parameterBy x, y, z and w, p, q, r converts to obtain, it is assumed that internal reference square
Battle array K is obtained by EXIF label;Then, in conjunction with internal reference:
Transformation matrix is obtained by image coordinate and world coordinates:
2D point and 3D point cloud are associated by this transformation matrix;Extract the evaluated assessment image for crossing camera matrix
FALoG characteristic point is as inquiry data set;For query characteristics point m (x, y), query characteristics are found by above-mentioned mapping algorithm
Closest feature, searches for established nearest data set features, and random forest passes through the inquiry in each decision node of test
Feature, finally inquiry reaches leaf node;The final mappings characteristics m' of random forest is the corresponding points of node maximum probability in m point,
Corresponding points m' has its relevant 3D point;For each 3D point, at least one corresponding image of database, each point projects to phase
The image of pass in conjunction with pixel difference as error function, and uses the error function iteration optimization attitude matrix, it is believed that different
The error of mapping point based on the sum of shape based on the pixel scale error of texture by being indicated, for each spy based on color
The scoring of posture of the sign using deviation ratio of the assessment based on features described above the most;
Related feature is as follows:
(1) based on the pixel characteristic of color
Color characteristic can indicate the surface properties of objects in images, and the fusion pixel using fusion single pixel and image-region is special
Sign method farthest expresses the color characteristic of single pixel;Two single pixel colors are distinguished using color distance function
Variation:
Wherein, p (x, y) and p'(x, y) it is two target points, R (), G () and B () they are respective corresponding R, G and B value;For
Region color feature, it is assumed that include target point in a region, it is assumed that have 5 × 5 pixels, for each RGB channel, area
The center in domain is target point;Extract target point (x, y) both horizontally and vertically on gradient value G (x, y):
Wherein, dhIt is the average value of two pixels of average value and right side of the difference target point between two pixels in left side;Central value
The other values in 48 horizontal and vertical regions are dv, pass through dhWhat same mode was calculated;For remaining four 2 × 2 pixel
Block, average down-sampling, finally, the pixel difference opposite sex based on color is:
Wherein, { 0,1 } δ ∈, if the difference of the down-sampled values of two blocks is lower than dpoint(p, p'), then δ is set as 0, is otherwise 1;
dregion(p, p') is used and dpoint(p, p') identical calculation method, dG(p, p') is the difference of gradient value, and η is regular factor;
(2) based on shape and based on the pixel characteristic of texture
As 32 × 32 pieces of center, Shape context feature is extracted, and calculates the Shape context of 25 sampled points, distributes block
Space logarithmic coordinates be divided into 12 × 3=36 part, expression shape feature difference indicates two dots of transformation using binarization
Shape diversity,Wherein, n=36, item ∈ { 0,1 }, item=1 mean that the train value of i block is greater than average value, right
It is divided into eight equal angular regions in the central point of texture, the circle that a radius is R;Calculate the average image in each region
Intensity, if average value is greater than center pixel value, the value in the region is set as 1, otherwise, is set as 0;Eight binary sequences
Textural characteristics are indicated with decimal number by converting;Finally, DtextureIt is determined by Hamming distances;
Finally, valuation functions E is the combination of features described above:
E(pi,pj)=a1Dshape(pi,pj)+a2Dtexture(pi,pj)+a3||Dcolor(pi,pj)||2
Wherein, ai∈ (0,1) indicates the weight of different item, DcolorThe canonical being considered as in this fusion diversity function
, DshapeAnd DtextureIt is connected, by k repetitive exercise the parameter in this function is assessed best,
Valuation functions describe posture estimation error inquiry two dimensional character and re-projection point of the priori data collection feature in characteristic point
Error is measured.
4. the camera Attitude estimation optimization method based on depth characteristic as described in claim 1, it is characterised in that in step 3),
It is described to be directed to instability problem present in the SLAM algorithm based on deep learning, propose a kind of multiple features boundling optimization algorithm
The specific method is as follows:
2D-3D mapping and key frame are stored by opening a new thread based on the deep learning SLAM system of priori knowledge,
Select those satisfactionsKey frame as boundling optimization (bundle adjustment, abbreviation BA) key frame
Set;Local BA optimizes all points seen by key frame;Observation point facilitates the optimization of constraint function and final posture, global
BA optimization is similar with part BA optimization, and only system will execute part BA between 30 frames, executes Global B A between 300 frames;
Table 2
Using after the SLAM system of optimization algorithm compared with other advanced SLAM systems it is as shown in table 2.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810878967.8A CN109035329A (en) | 2018-08-03 | 2018-08-03 | Camera Attitude estimation optimization method based on depth characteristic |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810878967.8A CN109035329A (en) | 2018-08-03 | 2018-08-03 | Camera Attitude estimation optimization method based on depth characteristic |
Publications (1)
Publication Number | Publication Date |
---|---|
CN109035329A true CN109035329A (en) | 2018-12-18 |
Family
ID=64648411
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810878967.8A Pending CN109035329A (en) | 2018-08-03 | 2018-08-03 | Camera Attitude estimation optimization method based on depth characteristic |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109035329A (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110009732A (en) * | 2019-04-11 | 2019-07-12 | 司岚光电科技(苏州)有限公司 | Based on GMS characteristic matching towards complicated large scale scene three-dimensional reconstruction method |
CN110033007A (en) * | 2019-04-19 | 2019-07-19 | 福州大学 | Attribute recognition approach is worn clothes based on the pedestrian of depth attitude prediction and multiple features fusion |
CN112101802A (en) * | 2020-09-21 | 2020-12-18 | 广东电网有限责任公司电力科学研究院 | Attitude load data evaluation method and device, electronic equipment and storage medium |
WO2021017314A1 (en) * | 2019-07-29 | 2021-02-04 | 浙江商汤科技开发有限公司 | Information processing method, information positioning method and apparatus, electronic device and storage medium |
CN112733761A (en) * | 2021-01-15 | 2021-04-30 | 浙江工业大学 | Human body state matching method based on machine learning |
CN112967296A (en) * | 2021-03-10 | 2021-06-15 | 重庆理工大学 | Point cloud dynamic region graph convolution method, classification method and segmentation method |
CN113299073A (en) * | 2021-04-28 | 2021-08-24 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for identifying illegal parking of vehicle |
WO2022252118A1 (en) * | 2021-06-01 | 2022-12-08 | 华为技术有限公司 | Head posture measurement method and apparatus |
CN116051723A (en) * | 2022-08-03 | 2023-05-02 | 荣耀终端有限公司 | Bundling adjustment method and electronic equipment |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170249491A1 (en) * | 2011-08-30 | 2017-08-31 | Digimarc Corporation | Methods and arrangements for identifying objects |
CN107833253A (en) * | 2017-09-22 | 2018-03-23 | 北京航空航天大学青岛研究院 | A kind of camera pose refinement method towards the generation of RGBD three-dimensional reconstructions texture |
CN108230337A (en) * | 2017-12-31 | 2018-06-29 | 厦门大学 | A kind of method that semantic SLAM systems based on mobile terminal are realized |
-
2018
- 2018-08-03 CN CN201810878967.8A patent/CN109035329A/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170249491A1 (en) * | 2011-08-30 | 2017-08-31 | Digimarc Corporation | Methods and arrangements for identifying objects |
CN107833253A (en) * | 2017-09-22 | 2018-03-23 | 北京航空航天大学青岛研究院 | A kind of camera pose refinement method towards the generation of RGBD three-dimensional reconstructions texture |
CN108230337A (en) * | 2017-12-31 | 2018-06-29 | 厦门大学 | A kind of method that semantic SLAM systems based on mobile terminal are realized |
Non-Patent Citations (1)
Title |
---|
HAN CHEN等: "Optimization Algorithm Toward Deep Features Based Camera Pose Estimation", 《ICIG 2017: IMAGE AND GRAPHICS》 * |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110009732B (en) * | 2019-04-11 | 2023-10-03 | 司岚光电科技(苏州)有限公司 | GMS feature matching-based three-dimensional reconstruction method for complex large-scale scene |
CN110009732A (en) * | 2019-04-11 | 2019-07-12 | 司岚光电科技(苏州)有限公司 | Based on GMS characteristic matching towards complicated large scale scene three-dimensional reconstruction method |
CN110033007A (en) * | 2019-04-19 | 2019-07-19 | 福州大学 | Attribute recognition approach is worn clothes based on the pedestrian of depth attitude prediction and multiple features fusion |
CN110033007B (en) * | 2019-04-19 | 2022-08-09 | 福州大学 | Pedestrian clothing attribute identification method based on depth attitude estimation and multi-feature fusion |
WO2021017314A1 (en) * | 2019-07-29 | 2021-02-04 | 浙江商汤科技开发有限公司 | Information processing method, information positioning method and apparatus, electronic device and storage medium |
US11983820B2 (en) | 2019-07-29 | 2024-05-14 | Zhejiang Sensetime Technology Development Co., Ltd | Information processing method and device, positioning method and device, electronic device and storage medium |
CN112101802A (en) * | 2020-09-21 | 2020-12-18 | 广东电网有限责任公司电力科学研究院 | Attitude load data evaluation method and device, electronic equipment and storage medium |
CN112733761A (en) * | 2021-01-15 | 2021-04-30 | 浙江工业大学 | Human body state matching method based on machine learning |
CN112733761B (en) * | 2021-01-15 | 2024-03-19 | 浙江工业大学 | Human body state matching method based on machine learning |
CN112967296A (en) * | 2021-03-10 | 2021-06-15 | 重庆理工大学 | Point cloud dynamic region graph convolution method, classification method and segmentation method |
CN113299073A (en) * | 2021-04-28 | 2021-08-24 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for identifying illegal parking of vehicle |
US12039864B2 (en) | 2021-04-28 | 2024-07-16 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method of recognizing illegal parking of vehicle, device and storage medium |
WO2022252118A1 (en) * | 2021-06-01 | 2022-12-08 | 华为技术有限公司 | Head posture measurement method and apparatus |
CN116051723B (en) * | 2022-08-03 | 2023-10-20 | 荣耀终端有限公司 | Bundling adjustment method and electronic equipment |
CN116051723A (en) * | 2022-08-03 | 2023-05-02 | 荣耀终端有限公司 | Bundling adjustment method and electronic equipment |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109035329A (en) | Camera Attitude estimation optimization method based on depth characteristic | |
CN111563442B (en) | Slam method and system for fusing point cloud and camera image data based on laser radar | |
Feng et al. | 2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud | |
Xia et al. | Geometric primitives in LiDAR point clouds: A review | |
CN111080627B (en) | 2D +3D large airplane appearance defect detection and analysis method based on deep learning | |
CN109410321B (en) | Three-dimensional reconstruction method based on convolutional neural network | |
Li et al. | A tutorial review on point cloud registrations: principle, classification, comparison, and technology challenges | |
CN113012122B (en) | Category-level 6D pose and size estimation method and device | |
CN109559320A (en) | Realize that vision SLAM semanteme builds the method and system of figure function based on empty convolution deep neural network | |
Yu et al. | Robust robot pose estimation for challenging scenes with an RGB-D camera | |
CN111709988B (en) | Method and device for determining characteristic information of object, electronic equipment and storage medium | |
CN110223351B (en) | Depth camera positioning method based on convolutional neural network | |
CN115546116B (en) | Full-coverage type rock mass discontinuous surface extraction and interval calculation method and system | |
Wu et al. | An object slam framework for association, mapping, and high-level tasks | |
KR101460313B1 (en) | Apparatus and method for robot localization using visual feature and geometric constraints | |
Dewan et al. | Learning a local feature descriptor for 3d lidar scans | |
Rubio et al. | Efficient monocular pose estimation for complex 3D models | |
CN104182747A (en) | Object detection and tracking method and device based on multiple stereo cameras | |
Wietrzykowski et al. | PlaneLoc: Probabilistic global localization in 3-D using local planar features | |
Jiang et al. | Leveraging vocabulary tree for simultaneous match pair selection and guided feature matching of UAV images | |
Zhang et al. | Improved feature point extraction method of ORB-SLAM2 dense map | |
CN114358133A (en) | Method for detecting looped frames based on semantic-assisted binocular vision SLAM | |
Álvarez et al. | Junction assisted 3d pose retrieval of untextured 3d models in monocular images | |
Hou et al. | Fast 2d map matching based on area graphs | |
CN111882663A (en) | Visual SLAM closed-loop detection method achieved by fusing semantic information |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WD01 | Invention patent application deemed withdrawn after publication | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181218 |