Texture organisation and mapping on Citrus sinensis point cloud

Abstract

In light of the current problems including coarseness, visible cracks, difficult data organisation, and the expensive memory requirements of the current texture methods, this paper mainly focuses on efficient organisation, linearised memory compression and seamless texture mapping between scanned Citrus sinensis images and point cloud information. Position and colour gradient based top-down splitting is proposed to simplify and organise the texture as texel descriptors to avoid both over-simplification and under-simplification. A Quadtree Morton and Z-order based linearised coding strategy is presented to compress the memory space of our texel descriptor based texture. A Gaussian Markov random field scheme was designed to smooth the ‘cracks’ between neighbouring texels. The simulated results on eight Citrus sinensises show that our simplification method reduces the texture memory requirements by 81.3 % over the original image, and 50 % over conventional simplification. The compression scheme also showed a 61.7 % improvement over the ordinary Morton code. Finally, the Gaussian Markov random field scheme makes the texture mapping smoother in comparison with other methods.

1 Introduction

Texture mapping has been most commonly used for rendering a photorealistic effects of a complex object surface as it improves the fidelity and simulation speed by replacing the details of the object model with an image [2]. Texture deformation can also be used to simulate details on the surface of doors, windows, branches, canopies, leaves and complex building surfaces. Both methods can reduce the modeling complexity while at the same time improving the efficiency and quality of the rendering [12, 14, 19].

With the increasing requirements of precision agriculture, the texture attributes of the Citrus sinensis surface will play an important role in cultivars selection, growth stage representations, product displays, the improvement of product prestige, fruit rank promotion, online sales, and foreign cooperation. At the same time, three-dimensional scanners provide the opportunity of acquiring “real size” and “real texture” from fruit in the field. Due to the complexity of the fruit-growing environment, it is difficult to scan point cloud data and images directly from the orchard. Therefore, a 3D scanner and a synchronised camera are employed to sample point cloud data and texture information respectively in our work, which will lead to the independence of the point cloud and image data, then may result in the dislocation between attributes and shape. This paper aims to solve these problems of coarseness, distortion, inefficient organisation and the expensive memory requirements existing in the Citrus sinensis texture mapping between an image and a simplified point cloud model.

1.1 Related work

In light of the widespread usage and thorough exploration of texture technology across all kinds of areas, representing the real Citrus sinensis texture morphology with accuracy, quality, and memory efficiency has been a long-time research focus in the areas of Computer Graphics, 3D reconstruction and image processing [13, 14, 16, 21]. Although the early texture mapping may benefit from the 2D texture information, our work can be categorised as a 3D shape based method which needs to cope with variances in digitization (e.g. colour, resolution, and accuracy), and settings (lighting, scaling, and background) for making Citrus sinensis texture mapping more applicable.

The early researchers mainly addressed 2D extension based methods, which only focused on the fine and smooth visual effects of the texture, and they were only suitable for texture synthesis on tiled artificial samples. G. Turk and L. Wei et al. extended two-dimensional texture synthesis methods to three-dimensional surfaces by using the pixel-growth method. According to the level within the pyramid structure, they provided the corresponding colour for each vertex based on the constraints of the vertex neighbourhood [21, 22]. Based on G. Turk [21], J. Zhang presented a texture synthesis algorithm with controllable orientation, scale gradient, and a texton mask based gradient process between the two different pieces of texture. Nevertheless, these methods have a limitation on the minimum size of texture, which cannot be less than the template size [26]. C. Soler implemented texture mapping of different scales by extending the block-based method to a three-dimensional geometric surface, for texture synthesis. However, the method was unable to control the change of direction in texture synthesis [18]. In conclusion, the methods cannot provide realistic appearance, colour and texture of fruit, which can vary greatly due to the diversity of the growing environments and the cultivars.

For the past few years, the introduction of 3D laser scanning in this area proved to be very useful, because of its invariance to the environment and the 3D measurements resulting in actual sized objects, and the depth information can be used to easily separate fruits from the background [8, 20]. At the same time, it can represent the morphology of a plant organ or fruit with rich detail, more flexibly, and can render the reconstructed object more reliably, because of non-topologic connections among the points [25]. However, in the early stage of 3D reconstruction, triangle mesh based texture synthesis methods, such as Cyclone6.0 and Riscan Pro, were first explored by the Leica and Riegl companies respectively [11], with the extra expense of converting point clouds into triangle meshes in advance. In these methods, Citrus sinensis textures were mixed by simply assigning the average colour value of the three vertices to the triangles. However, a high mesh density is needed in order to represent realistic Citrus sinensis surfaces; otherwise, a large area of the fruit surface may appear as a single colour due to the lack of triangle mesh detail in areas of little curvature. S. Magda proposed the block based fast rendering method with an extra requirement for the pretreatment of the texture information in advance. However the texture can only be blended artificially, and the process was only speed up by sacrificing quality [13]. The main problems of triangle mesh based texture mapping are that of the time-consuming and high memory space requirements of the actual Citrus sinensis points, which makes it impractical for texture mapping in large scan sets of , due to the following deficiencies [5]. First, the triangle count is twice that of the vertices; then, point, edge and the connectivity information needs to be saved for the triangle mesh; next, point clouds need to be pre-processed and converted into triangle meshes; last, the whole texture synthesis process has to be performed again whenever adding, deleting, or editing of any one of points. Fortunately, the direct point-based texture synthesis approach appeared later would address most of these issues .

Compared with a triangle mesh, point with both model position and texture properties made it seem easier to directly calculate the relationships between geometric attributes of the model and the final texture. The fact is, however, direct point based texture mixing was very difficult due to a lack of adjacency relations between the points [24] , which resulted in the point based texture mapping only being an auxiliary model editing function [1, 4, 10, 28]. X.C. Xiao proposed a global optimization algorithm, which synthesised the texture directly on a 3D sampled point-based surface, and the generated texture maintained the continuity and smoothness. They could also increase geometrical shade of the surface by adding some corresponding perturbation to the synthesis results based on the cell normal, according to the denseness of the discrete points or cells [23]. However, the quality of the texture was very difficult to improve, due to only considering the influence of reflection sampling on the texture and the limitations of the sampling points of each lumitexel [6]. Based on the idea of texture repair, Chan, RH et al. presented texture restoration and mapping methods using fractional order regularisation, which cannot generate feasible texture structure and cannot synthesise continuous, smooth visual effects, due to the limitation of using the local area [3]. Z. Xu gave a reconstruction of the Datong Yungang grottoes heritage landscape by directly mapping the corrected digital images to the scanned point clouds, but this simple and rough method may appear distortion due to only considering the 2D coordinates of the point clouds [27]. Some of the other point based texture mapping systems were based on extracted features and required the manual selection of features between the 2D images and the 3D points [14, 16]. However, The effective organisation and texture mapping is more complicated because the Citrus sinensis range data is separated from the fruit’s image data which is then mapped to a simplified, noisy, uneven, unorganised and incomplete point cloud, which can result in seams, gaps, overlaps or other distortions on the fruit texture surface and performance decline .

By comprehensive analysis and comparison of the different algorithms used for texture synthesis, we can come to the following conclusion. Early two-dimensional extension methods could not be applied to the texture mapping of the specific fruit surface due to only suitable for artificially sampled figure based texture synthesis. The initial point cloud based texture synthesis method mainly concentrated on the triangle mesh model, but it’s not applicable to Citrus sinensis fruits with scanned details on surface [13]. The direct point based method of texture blending proposed later can only be used for model editing , and cannot reflect the real texture characteristics of Citrus sinensiss surface [5]. Later some other methods have been developed, however, these result in a dramatic performance decline and seams, gaps, overlaps or other distortions on the generated texture surface when used for real fruits. Therefore, this paper will focus research on the efficient organisation, memory performance improvement and realistic mixing during texture mapping between scanned Citrus sinensis point clouds and image.

1.2 Contribution and organisation

This paper will provide the following 3 contributions to improve the Citrus sinensis texture organisation and real mapping on point clouds with a procedure shown in Fig. 1.

• (1). Efficient image organisation: In light of over-segmentation or under-segmentation existing in current methods, a top-down splitting method is presented to organise image into point-quadtree based efficient descriptor organization taking advantage of both gradient of position and colour.

• (2). Linearised coding and compression of texture: a Quadtree Morton scheme is presented for coding each descriptor to save the memory space and Z-order Linearisation strategy is employed to convert image topological structure into Linearisation for improving access performance of Citrus sinensis texels.

• (3). Authentic texture mapping: a Gaussian Markov random field was designed for texture mapping to protect the Citrus sinensis image from ‘seams’ or ‘cracks’ between neighbouring descriptors by repairing the edge of neighbour descriptors.

Fig. 1

Flow of the Citrus sinensis texture mapping

The remainder of this paper is organised as follows, first give the data acquisition scheme and analysis of the Citrus sinensis image in Section2. Then, we introduce the descriptor based texture splitting, organisation, coding, compression and mapping in Section3. In Section4, we discuss, analyze and compare the experimental results of our methods on Citrus sinensis textures. Finally the conclusions are given in Section5.

2 Capture and analysis of Citrus sinensis images

In order to make this research more practicable and valuable, the “Citrus sinensis Osbeck” cultivar of Citrus sinensis from Gannan, Zhejiang, China, is used. This is, to a certain extent, representative of Citrus sinensis texture features. Its image is of great value to horticulturalists in order to study the history and development, compare characteristics of different Citrus sinensis strains, and to breed better cultivars.

By taking a comprehensive consideration of the Citrus sinensis size, device scanning speed, price, device weight, data format and image performance, we chose a non-contact, portable laser 3D scanner, (PCP-300) with an inbuilt digital camera, as the capture device. The scanning coverage is 30 cm*20 cm, scanning distance is 35 cm, the depth of field is ±9 cm, max resolution is 1280*1024, and the accuracy is 0.025 mm. The real device dimensions are a maximum 11.15 cm*9.7 cm for the sides, 10.62 cm*9.74 cm for the bottom and 12.07 cm*10.44 cm for the top. Accordingly, based on the real size and surface curvature of the Citrus sinensis, the scanner performance, and using a 10 % overlap between two neighbouring images, we give the following scanning method. The side surfaces are taken from six angles, and the bottom and the top surfaces are taken once. We get eight groups of corresponding data with point cloud and image data in each. They are represented by the arrays Citrus (8) in Eq. (1) and Citrustex(8) in Eq. (2) respectively.

$$ \mathrm{Citrus}(8)=\left\{{\mathrm{s}}_1,{\mathrm{s}}_2,{\mathrm{s}}_3,{\mathrm{s}}_4,{\mathrm{s}}_5,{\mathrm{s}}_6,\mathrm{b},\mathrm{t}\right\} $$

(1)

$$ \mathrm{Citrustex}(8)=\left\{{\mathrm{s}}_1,{\mathrm{s}}_2,{\mathrm{s}}_3,{\mathrm{s}}_4,{\mathrm{s}}_5,{\mathrm{s}}_6,\mathrm{b},\mathrm{t}\right\} $$

(2)

These images of the Citrus sinensis are all saved as *.BMP and the texture dimensions are: side1 290*269, side2 309*269, side3 316*268, side4 299*274, side5 285*275, side6 309*274, bottom 301*276, and top 342*296. Part of the Citrus sinensis texture images are shown in Fig.2. According to related work, the corresponding information between transformed geometric attributes and the image data are stored separately and organised in their own local coordinate systems. This may lead to fruit texture displacement and large amounts of redundant data.

Fig. 2

Part of the original images of Citrus sinensis

3 Structuring organisation and authentic mapping of the Citrus sinensis texture

Due to been scanned from the outside obverse, each pixel of every Citrus sinensis image can be specified in a five dimensional space with two spatial coordinates and three colour components using Eq. (3),

$$ \left\{{\mathrm{Citrus}}_{\mathrm{image}}\right\}=\left\{{pix}_{ij}\right\}=\left\{{\mathrm{x}}_{ij},{\mathrm{y}}_{ij},{\mathrm{r}}_{ij},{\mathrm{g}}_{ij},{\mathrm{b}}_{ij}\right\}\kern0.5em \left(\mathrm{i}=1,2\dots \mathrm{N},\mathrm{j}=1,2\dots \mathrm{M}\right) $$

(3)

in which, N is the maximum rows and M is the maximum columns of each Citrus sinensis image.

3.1 Spatial position-colour based descriptor organisation for image data

Although the pixel position is specified as a two-dimensional coordinate, the image needs to be mapped on to a three-dimensional model, and it should be stored and organised using a three-dimensional specification. Each Citrus sinensis image is described in an x-y system, which is very different with the coordinate system of a scans’ convex surface, shown in Fig 3. For further processing convenient, the scans are first transformed from y-z plane to x-y by Eq. (4), in which, β is π/2, α is −π/2.

$$ \left[\begin{array}{c}\hfill \mathrm{x}\hfill \\ {}\hfill \mathrm{y}\hfill \\ {}\hfill \begin{array}{c}\hfill \mathrm{z}\hfill \\ {}\hfill 1\hfill \end{array}\hfill \end{array}\right]=\left[\begin{array}{cc}\hfill \begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ {}\hfill 0\hfill & \hfill \cos \upbeta \hfill & \hfill - \sin \upbeta \hfill \\ {}\hfill 0\hfill & \hfill \sin \upbeta \hfill & \hfill \cos \upbeta \hfill \end{array}\hfill & \hfill \begin{array}{c}\hfill 0\hfill \\ {}\hfill 0\hfill \\ {}\hfill 0\hfill \end{array}\hfill \\ {}\hfill 0\kern1.75em 0\kern2.25em 0\kern1em \hfill & \hfill 1\hfill \end{array}\right]\left[\begin{array}{cc}\hfill \begin{array}{ccc}\hfill \cos \upalpha \hfill & \hfill - \sin \upalpha \hfill & \hfill 0\hfill \\ {}\hfill \sin \upalpha \hfill & \hfill \cos \upalpha \hfill & \hfill 0\hfill \\ {}\hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\hfill & \hfill \begin{array}{c}\hfill 0\hfill \\ {}\hfill 0\hfill \\ {}\hfill 0\hfill \end{array}\hfill \\ {}\hfill \begin{array}{ccc}\hfill \kern1.5em 0\kern0.75em \hfill & \hfill \kern1.5em 0\kern1em \hfill & \hfill\ 0\hfill \end{array}\kern0.75em \hfill & \hfill 1\hfill \end{array}\right]\left[\begin{array}{c}\hfill {\mathrm{x}}^{\hbox{'}}\hfill \\ {}\hfill {\mathrm{y}}^{\hbox{'}}\hfill \\ {}\hfill \begin{array}{c}\hfill {\mathrm{z}}^{\hbox{'}}\hfill \\ {}\hfill 1\hfill \end{array}\hfill \end{array}\right] $$

(4)

Fig. 3

Local coordinates of Citrus sinensis scans with right obverse in x-axis, surface in y-z planes

in which[x^', y^', z^', 1]^T is the original coordinate of each point, and [x, y, z, 1]^T is the coordinate corresponding to the image.

Then the colour features of the Citrus sinensis in the Eq. (3) can be extended to six dimensional space by adding a z-component in the Eq. (5).

$$ \left\{{\mathrm{Citrus}}_{\mathrm{image}}\right\}=\left\{{pix}_{ij}\right\}=\left\{{\mathrm{x}}_{ij},{\mathrm{y}}_{ij},{\mathrm{z}}_{ij},{\mathrm{r}}_{ij},{\mathrm{g}}_{ij},{\mathrm{b}}_{ij}\right\}\kern0.5em \left(\mathrm{i}=1,2\dots \mathrm{N},\mathrm{j}=1,2\dots \mathrm{M}\right) $$

(5)

3.1.1 Colour gradient based splitting for Citrus sinensis images

Normally, the colour on the Citrus sinensis surface is relative smooth and the pixel positions are distributed uniformly, while the colour variance is not even. Due to the over-simplification or under-simplification of existing bottom-up K-neighbourhood methods on the non-uniform colour gradient of image, this paper designs a splitting based, top-bottom simplification strategy, which takes advantage of K-neighbourhood, quadtree, colour gradient and a max splitting level. By taking the position and colour value in Eq. (3), as input data, it performs a recursive process with a six-step splitting.

Before starting, we need to do some pre-processing. At first, although specified in three-dimensions, the pixels are only determined by a two-dimension position, which is used for our simplification. Then, because the conventional quadtree cannot split uneven colour variance [17], a point-quadtree is proposed for an axis-aligned partitioning at an arbitrary position inside pixels, the procedure is as follows.

• Step 1: the top-bottom simplification begins from an image closure P with a specified max level L, which encloses the whole data of a Citrus sinensis image with the max L value of six.

• Step 2: the colour gradient g_i , j against different neighbouring pixels is calculated by covariance analysis.

• Step 3: apply a condition judgment on the image closure P. P is no longer divided and skips to step6, when the number of its points is less than a prespecified threshold K or the splitting lever is larger than L or the colour gradient within the pixels is consistent, as shown in Eq. (6)

$$ {\mathrm{ming}}_{ji}\cdotp {\mathrm{g}}_{jk}>\delta \kern1.75em {\mathrm{g}}_{ji} = {\mathrm{g}}_{jk}=1 $$

(6)

• Step 4: generate arbitrary split coordinates t_split(x, y) by analysing the position distribution of pixels.

• Step 5: the image closure P is recursively decomposed into four uncrossed and compactly supported subsets B_j according to split coordinates t_split(x, y), then set the nonempty subsets as the input image closure P and return to Step3 to start a new splitting.

• Step 6: push the final image closure P from Step3 into a bucket B _T .

3.1.2 Organisation of the Citrus sinensis texture

Here, the Citrus sinensis texture is stored in bucket B_T and covered by a series of leaf nodes B_j, which are organized as texels with a set of pixels [7, 13] in Eq. (7).

$$ {\mathrm{B}}_{\mathrm{T}}=\left\{{\mathrm{B}}_{\mathrm{j}}\right\}=\left\{{pix}_{\mathrm{j},\mathrm{i}}\right\}\kern1.25em \mathrm{j}=1,2,\dots, \mathrm{n},\kern0.5em \mathrm{i}=1,2\dots, {\mathrm{m}}_{\mathrm{j}} $$

(7)

in which B_T is a Bucket covering the Citrus sinensis texture, B_j is the texels/leaf nodes in the Bucket, n is the number of texels in the Bucket, and j is the texel position in the Bucket; pix_j , i is the i^th pixel of j^th texel; m_j is the number of pixels in the j^th texel.

Then we calculate the attributes for each cell B_j, including the center position of the node in Eq. (8) and the size (radius) of the node in Eq. (9).

$$ {\hat{\mathrm{p}}}_{\mathrm{j}}=1/\mathrm{m}{\displaystyle \sum_{\mathrm{i}=1}^{\mathrm{m}}}{\mathrm{p}}_{ji} $$

(8)

in which, ${\hat{\mathrm{p}}}_{\mathrm{j}}\left({\hat{\mathrm{x}}}_{\mathrm{j}},{\hat{\mathrm{y}}}_{\mathrm{j}}\right)$ is the center position of B_j, p_ji(x_ij, y_ij) is the positon of pixel pix_j , i

In Eq. (10), the texture node descriptor τ_j is then defined in six-dimensions according to the position and colour attributes of the pixels within one texel.

$$ {\uptau}_{\mathrm{j}}=\left\{{\mathrm{R}}_{\mathrm{j},\mathrm{i}},{\mathrm{G}}_{\mathrm{j},\mathrm{i}},{\mathrm{B}}_{\mathrm{j},\mathrm{i}},{\hat{\mathrm{p}}}_{\mathrm{j}},{\hat{\mathrm{r}}}_{\mathrm{j}},{\mathrm{z}}_{\mathrm{j}}\right\} $$

(10)

3.2 Texel descriptor based linearisation coding and memory compression

The current space coding methods mainly target three-dimensional applications. The regular node of an octree use a tremendous amount of memory, because it needs at least 10 pointers, which point to the eight-child nodes, parent node, and the node attribute data. Although better at space compression efficiency, the depth-first and run length code are lower in query efficiency [9]. The Linear Octree encoding method, which appeared later, saves memory space by only concentrating on the leaf nodes, which store the address code and attribute values for that leaf node, implying the size and location information of the leaf node by taking the address code as the texel code. Nevertheless, the current Linear Octree algorithms are only applicable for the regular splitting Octree not a quadtree, also they have poor performance in query and storage efficiency when applied to Citrus sinensis descriptors due to incomplete splitting [15, 22].

It’s easy to know from the Morton coding process, that the Morton code is a procedure converting a 3D coordinate representation into a one-dimensional one [15, 22]. Based on Octree Morton, we present a quadtree Morton and linear compression process to improve query and memory efficiency. Actually, the current Morton code is a preliminary compression of the three-dimensional coordinate information by arranging it into one-dimensional code according to the z-y-x order.

(1) quadtree Morton coding

In our quadtree Morton code, the space size of the texture closure P is 2^L × 2^L after splitting, and the L-digit Morton code on any leaf node is shown in Eq. (11).

$$ {M}_q={q}_{L-1}\cdotp {4}^{L-1}+{q}_{L-2}\cdotp {4}^{L-2}+\dots {q}_0 $$

(11)

According to the splitting regulation in section3.1, the width of the texel data coding is 6 at most. Each leaf node has a 6-digit Morton code in Eq. (12)

$$ {M}_{q5}={q}_0{q}_1{q}_2\dots {q}_5={q}_5\cdotp {4}^5+{q}_4\cdotp {4}^4+\dots {q}_1\cdotp 4+{q}_0 $$

(12)

in which q _i is the node code of each splitting ,with the value range of 0…3.

The 6-level quadtree has a maximum unit size of 2⁶ × 2⁶ = 4096, which needs to be encoded with 6-digit quaternary code for each texel descriptor.

(2) linear compression

The developed Linearisation process on Quadtree Morton consists of three steps: the first is the data standardisation, the second is a space mapping from two-dimensional to linear space, and the last is a linearised coding.

• Step 1: For coding convenience, the texels coordinates should first be standardised by the means of scaling, translating, and view transformation etc. [15].

• Step 2: The Z-order space filling curve is employed to mapping the N-dimensional space points to one-dimensional space, due to its better locality, better retainability of spatial adjacency relations and a high conversion efficiency [22]. The data in each layer is mapped to a one-dimensional linear memory space in accordance with the sequence of planar Z-Order curves, one of which is shown in Fig.4. In the following steps, we will give the encoding of all textures starting from the 0-layer.

• Step 3: Linear encoding

Fig. 4

Partial Z-Order curve of S1

If Morton code is directly used as address code, it may produce some empty codes; as a result, some Morton code represents an empty node. As shown in Fig.5, the nodes 5,7, and 11 have taken up the storage location even though they are empty, and 12,13,14, and 15 are taking up four memory nodes even though they are actually one node. In this paper, we set up a two-dimensional Morton coding standard, in which we take the index of the data unit as a two-dimensional array subscript, and the valid Morton code in Fig.5 as array content. Part of the corresponding relationships between code and array c[i][j] are presented in Table 1. Morton is coded by quaternary code while being in strict accordance with the number of digits in the array, because it’s a two-dimensional coding on the 0-layer.

Fig. 5

Blank and incompletely split nodes

Table 1 Quadtree Morton code

In order to save memory, we calculate the code number from Eq.(12) according to the hierarchy of leaf nodes belonging to it, with the hypothesis that all original leaf nodes are at the bottom. Part of the generated code is shown in Fig.5. The process of converting Morton code from two-dimensions to one-dimension is devised as follows. By scanning all nodes with the Z-Order curve rules and analysing the relationships among them, we classify all texels into the following two cases and organise the code using different rules accordingly. At first, if the current texel is an empty leaf node, we move the next leaf node forward, such as the nodes with Morton code 6 and 11. Then, if the current texel is the first node of its parent with four empty consecutive nodes, such as 12,13,14, and15, that means the splitting process has been ended at its upper layer. The coding digit is assigned according to Eq. (12), whose value is based on the quaternary converted from the first node of the current layer by a 2-digit right shift (e.g. if the code value is 12 in decimal, 1100 in binary, then value will be 3 in quaternary after a 2-digit right shift). At last, if the current texel is a regular non-empty leaf node and without empty nodes around it, the appropriate coding digit is decided directly by Eq. (12), and all the code values are filled in Table 1. Here we have given an explanation on the implication of some codes like ‘03’ and ‘3’ in Table 1. The code ‘03’ means that, the current node is split two times, and it belongs to the ‘0’ position in the first level, and the ‘3rd’ position in the second level splitting. While code ‘3’ means it is only split once and no longer needs the second level splitting, which belongs to the ‘3rd’ position in its split-level.

Further investigation revealed that there still exists redundant codes due to the following two reasons. 1) Although there is no data to save, the empty nodes still occupy the position in the two-dimensional array (such as c [19] and c [2, 19], between c [12] and c [2, 12], and blank storage nodes c [12, 19], c [12, 19] and c [19] stored below the parent node c [12]). 2) Morton code itself has identified the splitting level and location, so the two-dimensional coding subscript no longer needs to indicate the location. For further space saving, we convert the code from the two-dimensional array of Table 1 to a one-dimensional array in Table 2, by taking advantage of the Z-Order sequence in Fig.4.

Table 2 One-Dimensional Morton code

The linear code is mapped to the disk storage for out-of-core expression, which is disk based method and provides the ability to decide which files should be loaded into memory using virtual storage management. This is achieved by rearranging the space order in the form of a dictionary corresponding to the layer number and spatial index.

3.3 Gaussian Markov random field based Citrus sinensis texture mapping

After Linearisation and coding, each texel is organised as descriptor τ_j with pixel number m_j. Normally, the colour values of texel B_j are mapped to the Citrus sinensis surface by averaging each of the colour components in Eq. (13),

$$ texel\left({r}_j,{g}_j,{b}_j\right)={\displaystyle \sum_{i=1}^{m_j}}\left({r}_{ji},{g}_{ji},{b}_{ji}\right)/{m}_j $$

(13)

As a result, visual seams may appear between different texels within one-image or neighbouring images. On the Citrus sinensis surface, current neighbour traits within a texel are similar to the Markov random field (MRF) model in the determining of the colour of the texels. The assumption is that the pixel colour depends on the probability distribution of the adjacent pixels’ values, which can make colour transition more smooth on the edges of different texels or images.

In this paper, the proposed texture mapping method can keep the texture fidelity by using a Gaussian Markov random field model to repair the cracks between texels.

At first, we take a texel in Eq. (14) as a set of two-dimensional grid points.

$$ {B}_j=\left\{ pi{x}_{j,1}, pi{x}_{j,2}\dots pi{x}_{j,{m}_j}\right\}\kern1.25em j=1,2,\dots, n $$

(14)

Then, we define a random field X in Eq. (15a), which can be seen as a Markov random field procedure on a two-dimensional image grid.

$$ X=\left\{{X}_{pix},pix\in {B}_j\right\} $$

(15)

In addition, the random variable X_pix should satisfy the contingent probability in Eq. (16), while implementing each time.

$$ P\left({X}_{pix}|{X}_r,r\in {B}_j,pix\ne r\right)=P\left({X}_{pix}|{X}_r,{X}_r\in \eta \right) $$

(16a)

$$ P\left({X}_{pix}\right)>0,\forall {X}_{pix}\in X $$

(16b)

in which, η is neighbourhood system on B_j, r is order of neighbourhood system with η ≤ r.

Third, define a neighbourhood system in Fig.6 (a) as a Gaussian Markov random field model, and cliques in Fig.6(b) as the neighbour relationship among grids within a neighbourhood system.

Fig. 6

Definition of the Gaussian Markov random field model and Neighbour relationship

At last, we use the Gaussian model in Eq. (17) as the contingent probability in Eq. (16a).

$$ P\left({X}_{pix}|{B}_j\right)=\frac{1}{{\left(2\pi \right)}^{\pi /2}{\left|V\right|}^{1/2}}\mathit{\exp}\left(-\frac{1}{2}{E}^T{V}^{-1}E\right) $$

(17)

in which, E is a Gaussian noise vector of zero mean, whose every element is the linear combination of the noise error between the center position ${\hat{\mathrm{p}}}_{\mathrm{j}}$ and the neighbourhood pixels.

4 Result and comparison

We carried out the Citrus sinensis texture organisation and mapping on a Hewlett-Packard HP Z420 Workstation (WM511EA, Intel Xeon CPU E5–1650 3.20GHz, 8.00GB RAM) with a GeForce GTX 560Ti NVIDIA Graphics card. The software used was OpenGL3.0 andVS2010.

4.1 Splitting and simplification of Citrus sinensis texture

Based on the colour gradient top-bottom simplification method in section 3.1, the original eight Citrus sinensis images in Fig.2 are split into sets of texels and are shown in Fig.7.

Fig. 7

the top-bottom simplification result on Citrus sinensis image with splitting resolution in row*column

From the above top-bottom splitting, it is obvious that the maximum produced texel resolution is 50*47, far less than the maximum number of original pixels 342*296, which greatly reduced the cell number and decreased the complex of the following steps.

We only list part of the final point-quadtree coordinates in Table 3, which is s1, s4 and t of the Citrus sinensis texture in Eq. (2).

Table 3 Part of splitting coordinates list of Citrus sinensis texture

4.2 Linearised compression coding on a citrus sinensis texture

Based on the above splitting result, all the texels B_j of Citrus sinensis are organised by the descriptor τ_j, then coded in Morton and compressed by Z-order Linearisation described in section3.2. In the simulation, a randomly selected texel from the Citrus sinensis top texture, in row 40, column 41, will be taken as an example. Its corresponding two-dimensional position in Table 1 is c[39][40], it’s then converted from a two-dimensional array to a one-dimensional linear storage array c[1990] in Table 2, according to the Z-Order curve in Fig.4 and rules in Fig.5. Lastly, according to the position sequence in Fig.4, six level splitting coordinates in Table 3 and regulation in Eq. (12), the quaternary Morton code is then generated in Eq. (18) for B₁₉₉₀, with 6*7 pixels.

$$ {M}_{1990}={q}_0{q}_1{q}_2{q}_3{q}_4{q}_5=220130 $$

(18)

We then give a comparison of the different methods with regards to memory requirements (in BYTEs) using eight Citrus sinensis textures. Before simplification, RGB colour components need to be saved for each pixel, and the memory pre_sim is row*column*3. In conventional quadtree splitting, all textures are divided into 2⁶ × 2⁶ nodes, which include 10 points and one attribute, the memory requirement is shown in Eq. (19)

$$ con\_sim={2}^6\times {2}^6\times \left({10}_{pointer}\ast 2+{3}_{attribute}\right) $$

(19)

While in our colour-gradient based simplification, the memory requirements of the Citrus sinensis image is shown in Eq. (20)

$$ colgra\_sim=ro{w}_{fig7}\times colum{n}_{fig7}\times \left({10}_{pointer}*2+{3}_{attribute}\right) $$

(20)

Based on our simplification, we apply quadtree Morton on each leaf node, which only stores address code, trait values, and the size and location information for a texel. The memory requirement is shown in Eq. (21)

$$ colgra\_sim\_ morton=ro{w}_{fig7}\times colum{n}_{fig7}\times \left({2}_{pointer}+{3}_{tra}+{4}_{width}+{4}_{heigh}+{2}_{position}\right) $$

(21)

By converting Morton code from two-dimensions into one-dimension, for further compression, each texel includes linear address code, trait values and a pointer to the next texel, with the memory requirements shown in Eq. (22)

$$ colgra\_sim\_com=ro{w}_{fig7}\times colum{n}_{fig7}\times \left(6/{8}_{add}+{3}_{tra}+{2}_{pointer}\right) $$

(22)

The memory required by all the Citrus sinensis textures in the five different stages is shown in Table 4.

Table 4 Memory space of citrus sinensis texture (Byte)

As seen in Table 4, for all the eight Citrus sinensis textures, the memory space of the original image ‘pre_sim’, in the 1st line, is highest because of the coding for each pixel. In conventional quadtree splitting, ‘con_sim’, in the 2nd line, declined dramatically by about 62.6 % over the ‘pre_sim’, but all images have the same space requirements because of regular splitting, which results in over-segmentation or under-segmentation. Our point-quadtree simplification ‘colgra_sim’, in the 3rd line, reduces the memory requirements by 50 % over the conventional quadtree splitting due to the consideration of the colour-gradient. Ordinary Morton code ‘colgra_sim_morton’, in the 4th line, saves a further 34.8 % based on our simplification result [15, 22]. At last, our Linearisation code scheme, in the 5th line, compresses the memory by 61.7 % over the ordinary Morton code. Finally, our simplification-code-linearization process in the 5th line saves texture memory by 95.3 % over the original image. The memory comparison is shown in Fig.8.

Fig. 8

Comparison of the memory of different procedures of using all the Citrus sinensis textures

4.3 Texture mapping between a citrus sinensis image and a point model

In this part, two previous methods are employed to estimate the attributes of each texel and map the Citrus sinensis images onto a model respectively. They are the colour components average [14] and the direct two-dimensional mapping method [27]. Then, our Gaussian Markov random field model algorithm described in section3.3 is used to calculate the texel attributes, map the texture and repair the cracks between texels by assigning a neighbourhood system. Figure 6 (a) shows the Gaussian Markov random field model, and Fig.6 (b) shows the cliques of the neighbour relationship among texels.

We give the simulation results of the different texture mappings, on Citrus sinensis surfaces, in Fig.9 to Fig.11. By comparative experiments of the texture mapping effects on Citrus sinensis scans using Fig.10, Fig.11 and Fig.9, it can be found that the colour components average is simple, coarse, and poorest at smoothing “seams” and texture discontinuities. The direct two-dimensional mapping method is better than using the colour components average, but still poor in dealing with the “seams” and discontinuities. Our modified Gaussian Markov random field model has achieved improved visual results by making the transitions smooth, and eliminating “gaps” between texels.

Fig. 9

Texture mapping of the Citrus sinensis by colour components average [14]

Fig. 10

Texture mapping of the Citrus sinensis by direct two-dimesional mapping [27]

Fig. 11

Texture mapping of the Citrus sinensis surface using a Gaussian Markov random field

5 Conclusion

In this paper, the proposed efficient texture organisation and mapping method has solved the issues with large memory space and smooth visual effects of the Citrus sinensis image. Experimental results also show that, the position-colour-gradient based top-bottom simplification method has dropped the leaf nodes by about 50 % over the conventional simplification. Then, our Z-Order based Linearisation code scheme has improved the memory compression by 61.7 % over the ordinary Morton code by converting texture code from two-dimensional to one-dimensional. Finally, the designed Gaussian Markov random field model algorithm has smoothed the ‘cracks’ between separate texels and has kept the texture authenticity when compared with previous techniques. Overall, the proposed texture organisation and mapping strategies have achieved a good tradeoff between space performance optimisation and visual effect for real texture mapping on Citrus sinensis surface.