GB2456301A

GB2456301A - Surface Representation from a Point Cloud

Info

Publication number: GB2456301A
Application number: GB0800263A
Authority: GB
Inventors: Didier-Madoc Jones; Robert Graves
Original assignee: GMJ DESIGN Ltd
Current assignee: GMJ DESIGN Ltd
Priority date: 2008-01-08
Filing date: 2008-01-08
Publication date: 2009-07-15
Also published as: WO2009087367A1; GB0800263D0

Abstract

This invention relates to a computer-implemented method of creating a representation of the surface of an object from a point cloud comprising data points. Each point defines a position on the surface of an object. The method comprises fitting trend lines (1003, 1005, 1007) to the data points, wherein a trend line represents a surface contour of the object. Intersection points (in subcells 1001d and 1001b) between trend lines are determined wherein an intersection point represents a corner feature of the surface of the object. An output comprising data defining the trend lines and intersection points to represent the surface of the object is produced. The object may be a building, typically in a model of a town or city.

Description

A METhOD OF CREATING A REPRESENTATION OF THE SURFACE

OF AN OBJECT

Field

The present invention relates to a method of creating a representation of the surface of an object from a point cloud. In particular, but not exclusively, the present invention relates to such a method where the representation is useable to produce a model of the object, such as a visual representation of the object on the display of a computer. More particularly, though again not exclusively, the present invention relates to such a method where the object is a rectilinear object and in particular a building, and typically where a model of a set of buildings is produced, for example a model of a town or city.

Background

A point cloud comprises data points, each point defining a position on the surface of an object. Typically, each point defines a measured position on the surface of the object. That is, typically the data points are collected directly from a real world object that is to be modelled. A device, such as a 3D scanner, is often used to collect the data on the shape or surface of an object for this purpose. Different technologies can be used as the basis for scanners depending on the target object and application. One such technique is known as LIDAR (Light Detection and Ranging). This is a remote optical technique for measuring the distance to a plurality of points on an object using light pulses. Time delay between transmission of a pulse of light and detection of the reflected light is used to calculate the distance. By scanning a plurality of points on the object, an array of data points is collected that represents the surface of the object. Other techniques exist for obtaining point clouds: for example, stereoscopic systems may be used to analyse the slight differences between two images of the same scene taken from different viewing positions in order to calculate the distance to the object.

A point cloud is usually expressed as a list or array of three-dimensional points.

Point cloud data, which is usually gathered from real-world measurements, usually contains some noise and/or uncertainty. A model based merely on the point cloud data (without processing or refining the data in some way) will be of poor quality due to this noise/uncertainty. For example, the edges of the surface of the object and edges of surface features of the object will appear poorly defined or ghost-like in the model. Furthermore, due to the size of the data set, processing point cloud data to produce a model can be slow and inefficient.

The present invention is made with the above considerations in mind.

Summary

According to an aspect of the invention, there is provided a computer-implemented method of creating a representation of the surface of an object from a point cloud comprising data points, each point defining a position on the surface of an object, comprising: fitting trend lines to the data points wherein a trend line represents a surface contour of the object; determining intersection points between trend lines wherein an intersection point represents a corner feature of the surface of the object; and producing an output comprising data defining the trend lines and intersection points to represent the surface of the object.

By using trend lines which represent surface contours of the object and determining intersection points between the trend lines, points which represent corner features of the object can be found. Since the output defines these points and the trend lines, a model of improved quality can be created.

Removing the noise by using trend lines and using these trend lines to determine well-defined corners in particular can improve the quality of the model. Furthermore, the output data is reduced in size when compared to the point cloud data and can be processed more quickly or efficiently to produce a model.

According to another aspect of the invention, there is provided a method of modelling an object on the display of a computer, comprising using the output of the method of creating a representation of the surface of an object to produce a model of the object.

According to another aspect of the invention, there is provided a method of modelling a set of buildings on the display of a computer, comprising using the output of the method of creating a representation of the surface of an object to produce a model of the set of buildings.

According to another aspect of the invention, there is provided a computer adapted to perform the method of creating a representation of the surface of an object.

According to another aspect of the invention, there is provided a computer readable medium having computer-executable instructions adapted to cause a computer system to perform the method of creating a representation of the surface of an object.

Other aspects and embodiments of the invention will be appreciated from the

following description and the appended claims.

Brief Description of the Drawings

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 shows a graphical representation of an example point cloud of data points defining measured positions on the surface of a building; Figure 2 is a flow diagram showing the steps of a method of creating a representation of the surface of an object from a point cloud; Figure 3 is a flow diagram illustrating the steps of another method of creating a representation of the surface of an object from a point cloud; Figure 4 is a schematic view of a building front illustrating surface contours; Figure 5 shows three orthogonal projections of data points of an example point cloud; Figure 6a shows an example principal line through the data points of the

example point cloud;

Figure 6b shows a profile of the surface along the principal line of Figure 4a; Figure 7 shows the example profile of Figure 4b divided up into cells of data; Figure 8a is a schematic showing some cells where a trend exists (fitted cells) and some cells where a trend does not exist (flagged cells); Figure 8b shows reducing the data in fitted cells to a single averaged point; Figure 8c shows an example multicell trend line; Figure 9a shows a flagged cell and adjacent multicell trend lines; Figure 9b shows multicell trend lines extrapolated into the flagged cell to identify an intersection point; and Figure 10 illustrates a method for identifying two corner points in a flagged cell.

Detailed Description of Embodiments

A point cloud comprises data points and is usually expressed as a list or array of three-dimensional points. Each point usually defines a measured position on the surface of the object. Figure 1 is a graphical representation in the form of a scatter plot of a point cloud of data points defining measured positions on the surface of a building, where each data point is plotted in three-dimensional space. In the figure, the outine of the surface contours of building -such as the edges of the windows -can be appreciated. Surface contours meet at corner features, although unless a data point happens to coincide with the position of the corner feature these are not shown in the figure.

In the described embodiments, the point cloud data points define positions on a surface of a building, such as the side of building.

Figure 2 is a flow diagram showing the steps of a method of creating a representation of the surface of an object (such as the side of a building) from a point cloud. The point cloud comprising data points, each point defining a position on the surface of the object. Referring to the figure, in step SI 00, trend lines are fitted to the data points of the point cloud. A trend line represents a surface contour of the object, for example an edge of a window recess of a building. In step S200, intersection points between trend lines are determined. An intersection point represents a corner feature of the surface of the object, for example a corner in a window recess of a building. In step S300, an output is produced comprising data defining the trend lines and intersection points to represent the surface of the object.

Figure 3 is a flow diagram illustrating the steps of another method of creating a representation of the surface of an object from a point cloud. Steps SilO, S 120 and S130 of Figure 3 illustrate one way of performing step S100 of the method of Figure 2. Similarly, steps S210 and S220 of Figure 3 illustrate one way of performing step S200 of Figure 2. Step 310 of Figure 2 corresponds to step 300 of Figure 2. The steps of the method of Figure 3 will now be described.

Step SilO In step SilO, the data points are processed to determine or identify one or more principal lines through the data. A principal line identifies an edge of a feature of the surface of the object.

A principal line lies on a plane applied to the surface of the object. In one embodiment, the plane is an x-z plane and the principal line is defined by a particular z-value (e.g. z = 7.50 metres). Whilst this plane will be referred to as an x-z plane, it will of course be appreciated that it can be labellçd alternatively, for example as an x-y or y-z plane. For simplicity though, the plane will be referred to as an x-z plane.

Typically, the or each principal line passes through corner features of the surface of the object. A principal line can be considered as a continuous line across the matrix of data points which passes through data points representing corners of features on the surface of the object, for example the corners of a window ledge or a door frame.

By identifying a line which identifies or passes along an edge of a surface features, subsets of the point cloud which contain the visually most significant features are identified; for example, corner features and the boundaries of surface features. Identifying these features can significantly reduce the amount of processing required to produce the representation of the surface of the object.

In one example, one or more principal lines are each determined by a four stage process as follows: In the first stage, a first plane (referred to in these examples as an x-z plane) is applied to the surface of the object. The x-z plane can be determined by identifying general trend directions in the data.

Figure 4 is a schematic view of a building front illustrating surface contours.

The figure also shows an example of a general trend direction. Referring to the figure an example building 401 is illustrated having a plurality of features, such as a recess 403 which accommodates a window. A general trend direction is a direction parallel to a significant number of contours on the surface of the object. One trend direction is depicted by arrow 405. Clearly, a second trend direction also exists, perpendicular to the first direction and this is depicted by arrow 407.

In this example, two perpendicular general trend directions are identified by, firstly, forming three arbitrary orthogonal projections of the point cloud data points. This is shown in Figure 5 in which the data points 501 are projected onto orthogonal planes 503, 505 and 507 to form projections 523, 525 and 527 respectively. The three projections show the general trend of the data on each plane. A straight line is fitted to each projection (shown in Figure 5 as lines 513, 515 and 517 respectively) and the two best fitting straight lines, here lines 513 and 515, define the best fit plane to the data. The two best fitting lines are also typically represent trend directions on the surface of the building since the edges of the dominant surface features, such as windows and doors, are usually parallel to one side or edge of the building.

The two best fitting lines 513 and 515 define the best fit plane. This plane is fitted to or represents generally the surface of the object. In embodiments, this plane is referred to as the first plane or the x-z plane.

In the second stage, a secondary line is identified.

At least one secondary line through the data and parallel to a general trend direction is chosen, for example a line parallel to the z-axis of the x-z plane.

The data points on the secondary line are analysed to determine whether the secondary line passes through the data representing a visually significant feature. In this example, this is achieved by assessing the deviation of the surface along the secondary line. More specifically, this is achieved by measuring the cumulative deviation of the surface data from the first plane (the x-z plane as identified in the first stage) along the secondary line. If the cumulative deviation is by at least a predetermined threshold, the chosen secondary line forms the basis of the third stage. That is, a suitable secondary line has been identified. If the cumulative deviation is lower than the predetermined threshold, a different secondary line is chosen and analysed.

Various techniques for selecting secondary lines may be used; for example, the secondary lines may be chosen at random. In an embodiment, the fiist secondary line passes through the centre (half way point) of the data, the second at a quarter way point and so on until a suitable secondary line is identified. The process of querying secondary lines is continued until one is found that passes through sufficient surface deviations. More than one secondary line may be identified and used in the third stage.

In one particular embodiment, the point cloud data points are represented as a two dimensional matrix in the first (x-z) plane where each data point is represented by an element in the two dimensional matrix. In the matrix, the position of an element defines the coordinates of a point in two dimensions (e.g. the x-and z-coordinates) and the value held in the element defines the value of the third coordinate of the point (e.g. the y-coordinate). The secondary lines can be considered as running along columns of the matrix and a secondary line can be identified by identifying a column having values which deviate from the plane by at least a predetermined amount, either cumulatively or as averaged values.

In the third stage, the data lying on the secondary line is analysed.

The data is analysed to identify the most significant features that the secondary line passes through. In an embodiment, this is achieved by looking for the feature points. The feature points are points on the secondary line where a change of the surface data along the secondary line is by at least a predetermined threshold. More specifically, if the difference between adjacent points is greater than a predetermined threshold, the coordinates of that point are identified for the fourth stage described below.

In one embodiment, in which the point cloud data points are represented as a two dimensional matrix in the first (x-z) plane with the values of the elements being the third coordinate value (the y-value), the values held in the elements along a dimension (a column or z-direction) of the matix are analysed to identify an element which holds a value (a y-value) that deviates by at least a predetermined amount from the value in an adjacent element.

In the fourth stage, the principal lines are defined.

The principal lines are defined as those lines which pass through one of the feature points and are perpendicular to the secondary line, and which lies on the first plane (the x-z plane). Thus, there can be one principal line for each line of feature points.

In one embodiment, in which the values held in the elements along a dimension (a column or z-direction) of a two-dimensional matrix are analysed to identify an element which holds a value (a y-value) that deviates by at least a predetermined amount from the value in an adjacent element, the position of that element (the z-value) is used to define the principal line.

Step S120 Referring back to Figure 3, step SilO is followed by step S120 in which a profile defined by the data points along the principal line is formed. This step is performed for each principal line in embodiments where there is more than one.

In one embodiment the profile is a two-dimensional profile in a second plane which is orthogonal to the first plane. In one embodiment, where the first plane is an x-z plane, the second plane is an x-y plane.

In one embodiment, in which the point cloud data points are represented as a two dimensional matrix in the first plane (x-z plane) with the values of the elements being the third coordinate value (the y-value), and in which the position of an element (the z-value) is used to define the principal line, the x-y values along the row of the matrix with that position (z-value) are taken to form the two-dimensional profile.

An example profile will be described with reference to Figures 6a and 6b.

Figure 6a shows a scatter plot 600 of the point cloud of measured data representing a substantially rectilinear building. The scatter plot 600 shows a number of features on the building such as recesses 602, which house windows Figure 6b shows a profile 650 across the surface of the building along a principal line 610 which passes along the edge of recesses 602 and 604. The profile contains discrete points 652 corresponding directly to the data from the source point cloud, but in only two-dimensions (here, x-y).

Step S130 At step S3 10 of Figure 3, for the or each principal line, trend lines are fitted to the profile defined by the data points along the principal line.

As can be seen from Figure 6b, the profile 650 contains a number of groups of data points, for example groups 654 and 656, which follow a trend; i.e. all the data points in the group fall on a continuous line which can be easily defined.

For example, the data points in group 656 all fall on a straight line 658 (which is extrapolated in the figure for the purpose of illustration only). Straight line 658 defines a trend line which represents a contour on the surface of the building. As can be seen from Figure 6b, a number of trend lines can be identified in the profile 650.

A number of different methods and approaches may be used to identify the trends in the profile and calculate the trend lines which best represents those trends. One example will be described with reference to Figures 7 and 8. As shown in Figure 7, a grid of cells 702 is applied to the profile 650, where the area covered by a cell contains a plurality of data points. Each cell containing data points is then analysed to determine whether a single trend line of a predetermined type can be fitted, within a predetermined tolerance, to the data points contained in that cell. In this example, the chosen trend line form is a straight line. A number of techniques may be used for fitting a straight line to a data set; for example, linear regression or an averaging process resulting in two points per cell and finding the equation of the straight line passing through both points. Figure 7 shows a first cell 704 in which a single straight line, labelled 706, can be fitted and a second cell 708 in which no single straight line can be fitted within a predetermined tolerance.

For certain profiles of certain objects, the trend in the data may extend across more than one cell. That is, adjacent cells can have a single trend line fitted, and it may be possible to fit one trend line across the adjacent cells. An example of this is shown with respect to Figures 8a -8c. Referring to Figure 8a, there are shown some cells, for example cells 801 and 802, for which a straight line can be fitted within the predetermined tolerance. These will be referred to as "fitted cells". There are also shown cells, for example cell 804 for which a straight line cannot be fitted within the predetermined tolerance.

These will be referred to as "flagged cells". In this embodiment, where there are adjacent cells that can have a single trend line fitted, one trend line is fitted to an averaged data point for each of those cells. That is, if a trend can be found in the data in one cell, all the data points in that cell are replaced by a single cell point. The single cell point is an average, in the direction of both axes of the profile, of all the data points in that cell. Referring to Figure 8b, single cell point 801a and 802a shows an average of the data points in cells 801 and 802 respectively. Once the data has been reduced in this fashion, the adjacent single cell points are analysed to see if a multicell trend line (a single trend line across more than one cell) can be fitted by any appropriate technique.

Figure 8c shows an example multicell trend line 806, which in this example is a straight line.

Since only two points in three dimensional space are needed to define a straight line, a number of noisy data points from the point cloud can be replaced by two data points which produce a smoother contour when reconstructed and accordingly provides a visually improved model.

Step S210 At step S210 of Figure 3, adjacent trend lines are extrapolated.

In one embodiment, in which trend lines are fitted to a profile defined by the data points along the principal line, a grid of cells is applied over the profile and two adjacent trend lines are extrapolated into a cell (a flagged cell) where no single trend line can be fitted to the data points within that cell.

Figure 9a shows an example profile 901 having several fitted cells, such as cell 903, and one flagged cell 905 where the original data remains (because no fit could be made). Two multicell trend lines 909 and 911 have been fitted to the single averaged points in the fitted cells. All other cells, such as cell 907, contained no data points and can be ignored. The multicell trend lines 909 and 911 from adjacent cells are extrapolated into the flagged cell 905.

Stei, S220 At step S220, intersection points between the extrapolated trend lines are determined or identified.

As shown in Figure 9b, two multicell trend lines 909 and 911 are extrapolated into a flagged cell 905 and the intersection identified. This can be done by ascertaining the equation of each line representing the averaged points and calculating the coordinate at which the equations are equal. The data contained in the flagged cell 905 is replaced by a single intersection point 915. If the trend lines from the adjacent cells do not intersect in the flagged cell, because for example two corners are in fact present in the flagged cell, a further analysis is conducted.

In this further analysis, a cell where no intersection point could be found is iteratively subdivided until two or more sub-cells containing corners are found.

This is achieved by fitting trend lines in the sub-cells and identifying those sub-cells which contain data points and in which a sub-cell trend line cannot be fitted within a predetermined threshold. . Once the sub-cells containing corners have been identified, the sub-cells are analysed in the same manner as before (extrapolation and intersection of adjacent trend lines) to compute new intersected corners. Figure 10 shows a flagged and divided cell 1001 containing two corners. The cell 1001 is subdivided until two or more data-containing sub-cells can be identified in which no sub-cell trend line can be fitted within a predetermined tolerance: these are shown as sub-cells lOOlb and lOOld in Figure 10. The data contained in sub-cells where a sub-cell trend line can be fitted is reduced to a single averaged point wherein the sub-cell single averaged point is the average of the data in that sub-cell in both directions. For example, single averaged point 1009 is the average of the data in sub-cell 10011. To identify the corner point in sub-cell lOOlb, the adjacent sub-cell trend lines 1003 and 1005 are extrapolated into sub-cell 1001 b and the intersection point found; this provides a good approximation to the location of the corner feature. The corner in sub-cell lOOld is, likewise, found by extrapolating adjacent sub-cell trend lines 1005 and 1007 into sub-cell bOld.

Finally, the data points originally in cell 1001 are replaced by the single averaged points in sub-cells lOOla, lOOlc, lOOle and lOOlf and the intersection points calculated in sub-cells lOOlb and bOld, which can be achieved by ascertaining the equation of each line representing the averaged points and calculating the coordinate at which the equations are equal Step S310 Finally, in step S3 10 of Figure 3, an output is produced (a new set of data) comprising data defining the trend lines and intersection points to represent the surface of the object. The output comprises the coordinates of the intersection points. In a particular embodiment, the output comprises a set of 3-dimensional coordinates which can be linked to produce a model of the object.

The 3-dimensional coordinates comprise the intersection points. In various embodiments, lines linking the 3-dimensional coordinates define simplified, but accurate, contours through the data. The output lines can be used to create a model of the object on the display of a computer, for example using commercial 3d modelling packages such as 3ds Max by Autodesk, Inc (San Rafael, CA, USA) or Rhino (available from McNeel, Seattle, WA, USA) or any package that can extrude surfaces between contour lines.

A computer such as a general-purpose computer can be configured or adapted to perform the described methods. In one embodiment the computer comprises a processor, memory, and a display. Typically, these are connected to a central bus structure, the display being connected via a display adapter. The computer can also comprise one or more input devices (such as a mouse and/or keyboard) and/or a communications adapter for connecting the computer to other computers or networks. These are also typically connected to the central bus structure, the input device being connected via an input device adapter.

In operation the processor can execute computer-executable instructions held in the memory and the results of the processing are displayed to a user on the display. User inputs for controlling the operation of the computer may be received via input device(s).

A computer readable medium (e.g. a carrier disk or carrier signal) having computer-executable instructions adapted to cause a computer to perform the described methods may be provided.

Embodiments of the invention have been described by way of example only. It will be appreciated that variations of the described embodiments may be made which are still within the scope of the invention.

For example, rather than applying an x-z plane to the surface of the object an x-y or a y-z plane can be applied.

Instead of defining straight trend lines to points, curved lines might be used, for example in the form of conic sections.

Claims

1. A computer-implemented method of creating a representation of the surface of an object from a point cloud comprising data points, each point defining a position on the surface of an object, comprising: fitting trend lines to the data points wherein a trend line represents a surface contour of the object; determining intersection points between trend lines wherein an intersection point represents a corner feature of the surface of the object; and producing an output comprising data defining the trend lines and intersection points to represent the surface of the object.

2. A method according to claim 1, comprising using the data points to determine a principal line which identifies an edge of a feature of the surface of the object; and fitting the trend lines to a profile defined by the data points along the principal line.

3. A method according to claim 2, wherein a first plane is applied to the surface of the object, wherein the principal line lies on the first plane, and wherein the profile is a two-dimensional profile in a second plane which is orthogonal to the first plane.

4. A method according to claim 3, wherein the first plane is an x-z plane and the principal line is defined by a particular z-value, and wherein the second plane is an x-y plane.

5. A method according to any of claims 2 to 4, comprising applying a grid of cells over the profile, wherein the area covered by a cell contains a plurality of data points.

6. A method according to claim 5, wherein fitting trend lines to the data points comprises determining cells for which a single trend line can be fitted and fitting a single trend line to the data points within those cells.

7. A method according to claim 6, wherein adjacent cells can have such a single trend line fitted, the method comprising fitting one trend line across the adjacent cells by fitting the one trend line to an averaged data point for each cell.

8. A method according to any preceding claim, comprising representing the point cloud as a two dimensional matrix wherein each data point is represented by an element in the two dimensional matrix, and wherein the position of an element defines the coordinates of a data point in two dimensions and the value held in the element defines the value of the third coordinate of the data point.

9. A method according to claim 8, wherein the method comprises using the data points to determine a principal line by analysing the values held in the elements along a dimension of the matix to identify an element which holds a value that deviates by at least a predetermined amount from the value in an adjacent element, and using the position of the element to define the principal line.

10. A method according to claim 9, wherein the dimension of the matrix is a column and the position of the element used to define the principal line is a row position of the element.

11. A method according to claim 10, wherein a plane is applied to the surface of the object, wherein the column is identified by analysing the data in one or more columns to identify a column having values which deviate from the plane by at least a predetermined amount.

12. A method according to any preceding claim, wherein the step of determining intersection points between trend lines comprises extrapolating adjacent trend lines and determining intersection points between the extrapolated trend lines.

13. A method according to any preceding claim, comprising using the data points to determine a principal line which identifies an edge of a feature of the surface of the object; fitting the trend lines to a profile defined by the data points along the principal line; applying a grid of cells over the profile and extrapolating two trend lines into a cell where no single trend line can be fitted to the data points within that cell.

14. A method according to any preceding claim, wherein the output comprises the coordinates of the intersection points.

15. A method according to any preceding claim wherein the object is a building.

16. A method of creating a representation of the surface of an object from a point cloud substantially as hereinbefore described with reference *to the accompanying drawings.

17. A method of modelling an object on the display of a computer, comprising using the output of a method in accordance with any preceding claim to produce a model of the object.

18. A method of modelling a set of buildings on the display of a computer, comprising using the output of methods in accordance with any preceding claim to produce a model of the set of buildings.

19. A computer adapted to perform the method of any preceding claim.

20. A computer readable medium having computer-executable instructions adapted to cause a computer system to perform a method of any preceding claim.