JP2002056374A - Homepage image reading method, program medium therefor, and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To read an image on a homepage from an optional desired observing point. SOLUTION: In this method, a user computer is connected to a web server through Internet to transmit a command of observing point change from the user computer reading an image data on the homepage to the web server, whereby a desired observing point image data can be generated by an interpolation processing by use of a stored basic data and transmitted to the user computer. The basic data can be formed by inputting a preliminarily taken image data to the web server and inputting characteristic quantity and corresponding point information.

Description

DETAILED DESCRIPTION OF THE INVENTION [0001] BACKGROUND OF THE INVENTION In a user computer
Homepage on web server via network
When viewing an
The present invention relates to an image browsing method that allows browsing from a point. [0002] 2. Description of the Related Art Conventionally, an image of an object is not displayed on a homepage.
When browsing image data,
Browsing. One frame per frame even for video
Received data and play back the received image continuously
It was realized by. Also, the direction from the mouse etc.
Even if you can change the viewpoint by input,
Internally, discrete image data prepared in advance is
Originally displayed. In addition, discrete image data
If you do not have the 3D shape data of the object,
It was projected and displayed on a two-dimensional plane. [0003] SUMMARY OF THE INVENTION In the conventional method,
View point based on discrete image data prepared in advance
When viewing an image of an object by changing the
The image data you want to see is not available
If not, I couldn't see it. In other words, the prepared visual
I could only see it from a point. Therefore, Arayu
If you try to prepare a different viewpoint, the amount of data will be huge,
It was not realistic from the viewpoint of communication speed. one
To be able to see from all perspectives
Is required by having three-dimensional shape data
It is necessary to project the image on a two-dimensional plane for display. Obedience
Therefore, the image of the object is internally converted to three-dimensional shape data.
Measurement of the actual object
Can generate 3D shape data if possible
However, 3D shape data is generated from video such as a photograph.
It was difficult to achieve. Further, on a network such as the Internet,
Online catalogs such as mail order sales using
Logs, online manuals for products and equipment, etc.
Documents have become popular, but the shortcomings of the documents
The problem of not being able to see and confirm the actual product, etc.
There is. Catalogs and manuals using websites
In, you can check from various perspectives
Although it has been devised in this way,
Display from the viewpoint desired by the user
It was difficult. [0005] [MEANS FOR SOLVING THE PROBLEMS]
In the first invention, a user's computer and a network
The web server that publishes the homepage on the network
Connect via a network and let the user
Data on the homepage via data and network
When browsing image data that is an image,
The user changes the viewpoint from the computer, and the user
Display image data of an object from any viewpoint
When the image data from any commanded viewpoint is
Take images of the object from multiple viewpoints in advance and perform predetermined preprocessing.
Using the basic data stored in the web server,
Is determined by an interpolation image generation method which is a predetermined interpolation operation.
It is characterized by having made it. Further, in the second invention, the basic data is
Image interpolation data for performing inter-operation and multiple basic images
The image interpolation data consists of multiple basic images.
Multiple corresponding points in the basic image
It is characterized by being calculated and obtained from the following. [0007] Further, a third aspect of the present invention is directed to a plurality of images in a basic image.
The corresponding points of at least 8 sets of corresponding point information
In addition, the predetermined feature amount is a plurality of corresponding features in the basic image.
It is characterized by line information. [0008] Further, a fourth invention provides a method for generating a plurality of basic images.
Points in the base image in the
The feature line information is input and set for a plurality of basic images.
It is characterized by the following. Further, in the fifth invention, eight or more sets of corresponding points are provided.
The image interpolation data obtained from the information is
Epipolar equation holds between two images
When A is a 3 × 3 camera calibration matrix and E is a 3 × 3 edge
P '₀, P '₁To image I₀, Image I₁
And the homogeneous coordinates in each image coordinate system of ## EQU3 ## F = (A^-1)^T EA^-1Then ... (1) [Equation 4] p '₁ ^TFp '₀= 0 (2) Solves the system of equations obtained by expanding equation (2).
Is determined using the 3 × 3 fundamental matrix F obtained by
Homography H which is a transformation matrix₀, H₁H is the inverse of₀
^-1, H₁ ^-1 Is a parallelized image obtained by
It is characterized by that. A sixth invention provides a method for generating an interpolated image.
Sets the interpolation point of the viewpoint corresponding to the command in the parallelized image.
By calculating, the parallelized interpolation image in the parallelized image
, And the obtained parallelized interpolated image is
To perform a predetermined operation equivalent to the homography conversion of
And obtain an interpolated image of a desired viewpoint.
You. Further, according to a seventh invention, a web server includes:
Receives commands from the user computer and stores the stored
Using this data, the user can
Generate image data from any desired viewpoint, and
It is transmitted to a user computer. According to an eighth aspect of the present invention, the web server includes:
Receives commands from the user computer and stores the stored
This data is sent to the user computer and
Computer receives the basic data from the web server and
Depending on the image generation method, from any viewpoint desired by the user
Is generated. [0013] Further, a ninth invention provides a homepage image.
The web server used for browsing the website
Homepage publishing means and user computer
Hand that connects to the computer and is accessible via the network
And a command receiving unit for receiving a command from the user computer.
Communication means, basic image input means for inputting a basic image, and
Input a plurality of corresponding points in the main image or predetermined feature values
Corresponding information input means, input basic image, and corresponding information
Image interpolation data calculation means for calculating image interpolation data from
And basic data storage means for storing basic data;
Using the received basic data, the
Image data from any viewpoint according to commands from the data
Means for generating an interpolation image, and
Image data transmission to send data to the user computer
Communication means. Further, a tenth invention is directed to a homepage image.
The user computer used for the image browsing method is a web
Connect to the server so that it can be accessed over the network
Web server for connecting means and user computer commands
Command sending means to send to the web server
Image data from any viewpoint desired by the user
Image data receiving means for receiving the image data
And [0015] Further, an eleventh invention is directed to a homepage image.
The web server used for the image browsing method is the homepage
Means for publishing web pages and user computer
Connection means for making an accessible connection to the
Input basic image input means and multiple correspondences in the basic image
Point, or corresponding information input means for inputting a predetermined feature amount,
Image interpolation data from input basic image and corresponding information
Image interpolation data calculation means for calculation and basic data are stored
Basic data storage means and a user computer
Command receiving means for receiving commands, and stored basic data
Data transmission means for transmitting data to the user computer;
It is characterized by having. Further, a twelfth invention is directed to a homepage image.
The user computer used for the image browsing method is a web
Connect to the server so that it can be accessed over the network
Web server for connecting means and user computer commands
Command sending means to send to the web server
Basic data receiving means for receiving basic data
Image data from any viewpoint using the basic data
Interpolated image generating means for generating data
I do. Further, a thirteenth invention is directed to a homepage image.
Web server readable program used for image browsing
On the recording medium that stores the program,
Procedures for publishing home pages and user computers
Connection procedure for making an accessible connection to the
Command reception procedure for receiving commands from the
Basic image input procedure to input and multiple correspondences in the basic image
A corresponding information input procedure for inputting a point or a predetermined feature amount,
Performs image interpolation data from the input basic image and corresponding information.
Image interpolation data calculation procedure and basic data
Using the basic data storage procedure and the stored basic data
Any viewpoint according to the command from the user computer
An interpolated image generation procedure for generating image data from
Send generated image data to user computer
Image data transmission procedure to perform
You. Further, a fourteenth invention is directed to a homepage image.
User computer readable for image viewing method
On a recording medium that stores
Connection procedure to connect to the
Command transmission procedure for transmitting computer commands to web server
And the viewpoint desired by the user sent from the web server
Data receiver that receives image data from
And the order. Further, a fifteenth invention is directed to a homepage image.
Web server readable program used for image browsing
On the recording medium that stores the program,
Procedures for publishing home pages and user computers
Enter connection procedure and basic image to be accessible
Basic image input procedure, multiple corresponding points in the basic image,
Or a corresponding information input procedure for inputting a predetermined feature amount, and inputting
Calculates image interpolation data from the extracted basic image and the corresponding information
Image interpolation data calculation procedure and the basis for storing basic data.
This data storage procedure and the command from the user computer
The command receiving procedure to receive and the stored basic data
Having a basic data transmission procedure for transmitting to the computer
It is characterized by the following. Further, a sixteenth invention is directed to a homepage image.
User computer readable for image viewing method
On a recording medium that stores
Connection procedure to connect to the
Command transmission procedure for transmitting computer commands to web server
And receive the basic data sent from the web server
Using the basic data reception procedure and the received basic data,
Interpolated image generation that generates image data from any viewpoint
And a synthesizing procedure. [0021] According to the first aspect of the present invention, the user's computer
Open your homepage on your computer and network
Web server is connected via a network,
When the user connects to the user computer via the network,
Check the image data, which is the image of the object on the homepage
When browsing, the user receives a command from the user computer
Image data of the object from any viewpoint desired by the user
Can be displayed. Any commanded
Image data from the viewpoint is obtained by pre-
Basic data captured from the viewpoint and stored in the web server
Using the interpolation image generation method which is a predetermined interpolation operation
As required. Therefore, take pictures from multiple viewpoints in advance.
By preparing at least two images,
Of viewpoints within the range of differences between viewpoints in multiple images
Images can be obtained by a predetermined interpolation calculation
Therefore, do not prepare an image for each minute change like a moving image.
Of the range of differences between viewpoints in the prepared image
If it is a point, you can get an image from the desired viewpoint
Become like Therefore, if necessary,
Obtain images from all directions by preparing multiple images
Will be able to do it. Furthermore, based on the captured image
Video camera to obtain the image of the desired viewpoint by interpolation calculation
Image or a photographed image
Even if there are images from multiple viewpoints,
Interpolation operation to obtain image data
become. In other words, three-dimensional shape data
No longer needed. This allows a large amount of image data
It is no longer necessary, and it is difficult to obtain from photos etc.
This eliminates the need for the three-dimensional shape data of the object. Further, according to the second invention, the basic data
Is the image interpolation data for performing the interpolation
This image data consists of several basic
Multiple corresponding points in the basic image in the image, or
Because it is characterized by being calculated and collected by the collection,
Conventional imaging, such as markers that were captured together
Eliminates the need for indicators and allows for
Internal parameters such as internal parameters and focal length are also required
No longer. Further, according to the third invention, the basic image
Of multiple corresponding points is at least 8 pairs of corresponding point information
And the predetermined feature amount corresponds to a plurality of corresponding features in the basic image.
Because it is feature line information,
And specify 8 sets of corresponding points, and
Is specified in the second aspect, the image interpolation data
Can be calculated. Further, according to the fourth invention, a plurality of basic
Multiple corresponding points in the basic image or multiple features in the image
Marking line information is input and set for multiple basic images.
Because, for example, it is displayed on the display device
Input device such as a mouse for two basic images
Click the corresponding point and drag the feature line
Can be specified and input.
You. Further, according to the fifth invention, the given
Epipolar equation holds for two of the basic images
By setting 8 or more sets of corresponding point information,
Get the fundamental matrix and the homography of each image
To be able to get the desired interpolated image
Can obtain a parallelized image which is the basic image data of
Become like Further, according to the sixth aspect, the interpolation image generation
The generation method is based on the viewpoint corresponding to the command between the parallelized images.
Find the interpolation point and obtain the parallelized interpolated image in the parallelized image
To the obtained parallelized interpolated image.
A predetermined performance equivalent to the homography conversion of the corresponding viewpoint
Calculation to obtain an interpolated image of the desired viewpoint.
become able to. That is, the above-mentioned eight corresponding points
And input a feature line to obtain an interpolated image of the desired viewpoint
Will be able to do it. Further, according to the eighth invention, a web server
The server receives the command from the user computer and stores the command.
User data by the interpolation image generation method using the basic data
Generate image data from any viewpoint desired by the user
And send it to the user's computer.
With a large amount of moving image data
Data need not be sent. Further, according to the ninth aspect, the web server
Is stored by a command from the user computer.
Basic data can be sent to the user computer
Become so. The user computer sends the transmitted
This data can be received. further,
By the interpolation image generation method using the received basic data,
Generate image data from any viewpoint desired by the user
Can be achieved. This allows each viewpoint
User computer without receiving image data
Data can generate interpolated images from any viewpoint
Become like If you are not satisfied with the communication speed,
Interpolation on user computer only by receiving data
High-speed image data
Can be expected to change. Further, a web server according to the ninth invention
The web server readable program according to the thirteenth invention
Install and use the program, web server
Will be able to publish their homepage,
Accessible via user computer and network
Noh can be connected. Further, basic image input means, for example,
The basic image is imported to the web server using the
This can be used as copy data. Internal data and
Of the converted basic image by the corresponding information input means.
It becomes possible to input and set correspondence information between main images. Sa
Furthermore, image interpolation data calculation based on the input correspondence information
Calculate and record the image interpolation data according to the procedure in the means.
It can be stored in storage means. Above
In the range of the difference between the viewpoints of the input basic image.
Pre-processing to obtain image data from the desired viewpoint
become able to. The basic data is information of a captured image.
Basic image and preprocessed image interpolation data.
Basic images include photos, video shot images, and digital cameras.
An image of a camera, etc.
At least two images. This image is
No position, orientation, lens information, etc. are required. Further, a command from the user computer is
Receives and responds to commands from the received user computer
Image data from any viewpoint based on basic data
Interpolate and send to user computer
Become so. Further, the user computer of the tenth invention
To the user computer according to the fourteenth invention.
By installing and using possible programs, users
The computer sends the web server according to the ninth aspect to the web server.
Web server readable program according to the thirteenth invention
Web server and network to install and use
To be accessible via the network. Ma
Also, send the command of the user computer to the web server.
Can be sent from the web server
Image data from any viewpoint desired by the user
You will be able to receive. Furthermore, a web server according to the eleventh invention
A web server readable program according to the fifteenth aspect of the present invention.
Install and use the program, web server
Will be able to publish their homepage,
Accessible via user computer and network
Noh can be connected. Further, basic image input means, for example,
The basic image is imported to the web server using the
This can be used as copy data. Internal data and
Of the converted basic image by the corresponding information input means.
It becomes possible to input and set correspondence information between main images. Sa
Further, based on the input correspondence information,
Calculate and store the image interpolation data according to the procedure in the stage
It can be stored in the means. Above
Before obtaining image data from the desired viewpoint
Processing will be possible. Further, a command from the user computer is
Receives and responds to commands from the received user computer
Image data from any viewpoint
Data to the user computer for data generation
You will be able to send. Further, the user computer according to the twelfth invention
To the user computer according to the sixteenth invention.
If you install and use a possible program,
The computer sends the first to the web server according to the eleventh invention.
Web server readable program according to the fifth invention
Web server and network using and installing
Will be able to connect accessible via
You. User computer for changing the viewpoint of images
To the web server and send from the web server
Basic data to be received.
In addition, using the received basic data, from any viewpoint
Image data can be generated.
As a result, the interpolation image is obtained on the user computer side.
Can be generated. As a result, the image
The basic data is compared to the case where
Since the amount of data required is small,
The advantage is that image data from a point can be obtained at high speed.
You. [0039] BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described with reference to specific examples.
It will be described based on the following. FIG. 1 is a diagram showing a connection relationship.
The computer 10a of the user 1 is provided, for example, at home.
Have been. User 1's computer 10a is
Line 8, Internet service professional to which the user subscribes
A home page is provided via an Internet network 7a including a binder.
Connect to the web server 20 that has published the page, and on the screen
The home page is displayed. Here, the network
Will be described using the Internet as an example. Network
As a wide area network (hereinafter WAN)
Describe), corporate local area network (below)
Network).
No. The image data to be browsed is stored on a web server.
20 is the image on the published homepage,
There are no restrictions on the home page. For example, client
The homepage of the project may be published on behalf of the public. Also,
If necessary, the computer 70a of the client 1
Connect to the web server 20 via the Internet network 7b
Have been. The client is the web server 20
Users who are requesting their home
It is the. As a client, I want to sell products
Sales company or general individual. Sa
In addition, the web server 20 has LAN5, WAN, etc.
The management terminal 9 is connected via the management terminal 9. Also, a web server
The connection method to the server 20 is the user 10's computer 10c.
LAN6, WAN without going through the Internet network
It may be connected via such as. In the above and subsequent explanations, for simplicity,
User's computer, client's computer
In the typical example, we will explain one device.
User computer and client computer
Data can be connected to multiple units.
Is a system where multiple units are connected and installed.
Are common and will not be described. A method for browsing homepage images according to the present invention
Of the user computer of user 1
It is installed and used on the computer 10a. this
These programs include, but are not limited to, for example,
It runs under a general-purpose OS such as Windows 2000 (registered trademark).
Be moved. The program of the web server 20 is
Installed on the computer of the web server 20 and used
Can be These web server 20 programs are
Without limitation, for example, Windows 2000
It is started under a general-purpose OS corresponding to a server. The home page published by the web server 20
The content on the page is the home created by each client
The page is sent to the web
Uploaded to the server 20 and published as a proxy
be able to. Also, a class that cannot create a home page
To the client, the individual created by the web server 20
Website for clients
Is also good. In this case, for the image data of the image,
In the client computer 70a, the scanner
Web server based on image data input from
At 20, the data may be converted to basic data. [0044] Such a home page is input by the user.
Via the tannet network 7a and the computer 1a of the user 1
You are browsing. For example, on the displayed homepage
If you want to change the viewpoint of the image of
An arrow indicating that you can change the viewpoint of the mouse pointer
It is displayed on the image instead. From the mouse along the arrow
Click input changes the viewpoint of the image. Next, the apparatus of this homepage image browsing method
The operation of the device will be explained with reference to the block diagram
I will tell. For detailed operation of each computer,
It will be described later according to each flowchart. The following description of the operation is based on the Windows OS.
Is assumed. In addition, browsing the website
Internet Explorer, but not limited to
Browser software is installed in advance. Ma
User 1's computer 10a, web server 20
Each page of this website browsing method
The program is installed in advance and the web server
20 is also set in advance as necessary.
You. Used in this homepage image browsing method
The computer 10a of the user 1 has a control device 50, an input
Device 2 (60), external storage device 2 (61), display device 2
(62), which is generally a personal computer
A computer or the like is used. User 1's computer 10
a connects the Internet network 7a by the connecting means 2 (55).
It is connected to the web server 20 via the Internet. Control device 50
Is the viewpoint direction of the image input from the input device 2 (60)
Transmission means 2 (51), command transmission means 52
And the web server 20 via the connection means 2 (55)
To the web server 20 and the image of the image transmitted from the web server 20.
Means for receiving image data (53), an image data receiver
Received via stage (54a) and displayed on display 2 (62)
Show. The external storage device 2 (61) is HD, CD-RO
M, MO, FD, etc., and their devices. Ma
In addition, the external storage device 2 (61) stores an image as necessary.
The data may be stored. Input device 2 (60)
The display device 2 includes a keyboard, a mouse, and the like.
(62) is composed of a CRT, a liquid crystal panel, etc.
You. The web server 20 has a control device 30 and an input
Device 1 (40), external storage device 1 (41), display device 1
(42), using a server computer.
It is. The web server 20 uses the connection means 1 (39)
User 1 via the Internet network 7a.
Data 10a. Input device 1 (40) is a key
Board, mouse, scanner, video capture, digital
It consists of a digital camera, video, pen tablet, etc.
Input various data to create basic data.
U. The external storage device 1 (41) includes an HD, a CD-ROM,
MO, FD, etc., and their devices. As needed
The image data is also stored. Display device 1 (4
2) is composed of a CRT, a liquid crystal panel, etc.
Display image data, create and confirm basic data
U. The control device 30 corresponds to the basic image input means 33A.
Input from the input device 1 (40) by the information input means 33B.
Image line data and feature line as feature
From the information and the corresponding point information, the image interpolation data calculation means 3
2B, the basic data is calculated, and the basic data storage means 3
Store it in 2. User 1 via the Internet network 7a
Change of image transmitted from computer 10a
The command is received by the receiving means 1 (34) and the command receiving means 35.
Received by the interpolation image generation means 31 in accordance with the command.
The interpolation image is stored in the basic data stored in the basic data storage means 32.
Data, the homepage publishing means 38, the sender
Step 1 (36), the image data transmitting means 37a
Transmitted to the computer 10a of the user 1. Next, the web server 20 and the user 1
Flowchart for processing procedure of computer 10a
This will be described in detail with reference to FIGS. Book floater
The main section mainly explains the present invention in detail,
Web server 20 and computer 10a of user 1
Since other processing of is known, it is not described here.
No. First, in the web server 20, the home page
If you want to create the data
Device 1 (40) and basic image input means 33A,
The information input means 33B uses the information
Basic data is created by interpolation data calculation means 32B
Then, it is stored in the basic data storage means 32A. Basic Day
The details of creating the data will be described later.
A brief process will be described. Next, in step 106
To enter data other than the basic data,
Create a page. In step 108, the user 1
Check whether there is a connection request from the computer 10a.
Click. If there is a connection request, step
110 to the user 1's computer 10a
Perform the process of publishing the page. In step 112,
A maintenance maintainer whose web server 20 is determined to be down
Web server 20 is generally continuous, except for
In operation. Next, from the computer 10a of the user 1
In step 300, the connection request to the web server 20 is reset.
When the quest is performed, in the web server 20,
In step 108, it is determined that there is a connection request.
It is. The connection request here is, for example,
For example, a PPP connection based on a connection up or the like. Step 3
02, step 304, and step 202
The homepage of the server 20 is sent to the computer 1 of the user 1
0a can be browsed. Step 204,
In step 306, the image including the image data of the image
Is downloaded to the user 1
The home page of the web server 20 is displayed on the computer 10a
Is done. In step 308, the image data
If it is determined that there is an input to change the viewpoint of
The finger that changes the viewpoint from the command transmission unit 52 by the
An order is sent. In the web server 20, the steps
206, the command is received by the command receiving means 35.
If it is determined that the interpolation image has been
In the generation means image 31, the basic data storage means 32
Image interpolation processing is performed using the stored basic data.
In step 210, the image data transmitting means 37
The image data is transmitted by a. Sent
The image data is sent to the image data receiving means (54a).
(Step 314). To step 316
Request for disconnection from the computer 10a of the user 1.
Until there is an est, it stays in the computer 10a of the user 1.
To browse the homepage. User 1's compilation
It is determined that there is a disconnection request from the computer 10a
Then, in step 318, a cutting process is performed. We
The server 20 also disconnects at step 212.
Is determined to be present, and in step 214
A cutting process is performed. Next, another embodiment of the present invention will be described with reference to FIG.
First, according to the lock diagram and the flowcharts in FIGS.
The emphasis is placed on the differences from the described invention, and
Omitted. Control device for computer 10a of user 1
At 50, the basic data is generated because the interpolated image is generated.
Data receiving means 54b for receiving the basic data
It further has a receiving means 54b. When browsing images
At the user 10's computer 10a,
A change command is transmitted by the command transmitting means 52 (step
510). In step 406, the web server
At 20 it is determined that the command has been received.
The data transmitting means 37b and the transmitting means 1 (36)
This data is transmitted (step 408). Step 5
12. In step 514, the basic data receiving means 5
4b receives the basic data. To step 516
Here, the interpolation image is generated by the interpolation image generation means 56.
It is. Next, the creation of basic data will be described with reference to FIG.
This will be described according to a flowchart. In step 600,
Capture image data by input device 1 (40)
No. Image Image data consists of a single scene or object.
At least two or more images viewed from several viewpoints. This
The image data here is used for photos and digital cameras.
Images captured, such as video, video camera video, etc.
You. Here, two images are given for simplicity
The case will be described. More images are given
In this case, the viewpoint is continuous between the given images.
As you can see, by selecting images and performing similar processing
Perimeter and better view of the scene or object
Interpolation and generation of image data from the viewpoint
Images can be provided. First, a step is performed on the given two images.
In 602, a pen tablet, mouse, keyboard
Input 8 or more corresponding points of the input image using
You. In other words, in two images, the same part is scored 8 points or less.
Enter above. Determine the matrix value based on the input 8 points
To do so, no matter which three points are taken,
You have to select the corresponding points. Next, in step 604, the feature quantity
That is, the user inputs feature line information. Here, the feature line information and
Is the outline of the object being imaged in the image
is there. At this time, the entire feature line is captured in both images.
Is desirable. In addition, the images will be discontinuous
Such as the scene being shot, the ridge of the object
Lines such as lines, outlines, and pattern borders are input as feature lines.
It is desirable to be. At this time, the characteristic line is a straight line
However, for the contour of a curve, input by dividing into straight lines
Corresponds approximately. Enter in step 606
A base matrix is calculated from the corresponding point data obtained. Less than
In addition, the position of the two basic images when calculating the fundamental matrix
Describe the relationship, the definition of each matrix and transformation, and the operation method.
You. FIG. 11 schematically shows the positional relationship of each coordinate system.
Show. Two images I of the same object viewed from different viewpoints₀,
I₁Is given, the three-dimensional one from two perspectives
Point B '= (X, Y, Z, W)^TCan be estimated
You. B ′ is a three-dimensional space coordinate value B = (X, Y, Z)^TSame
Next coordinates. In the following description, unless otherwise specified
In this case, the coordinate α ′ represents the simultaneous coordinate of the coordinate α and the coordinate α.^TIs
The transposition of the target α is given. In each image coordinate system
Let the two-dimensional coordinates of B be p₀, P₁, And its homogeneous coordinates are p '₀,
p '₁, And. The two-dimensional coordinates of B in each viewpoint coordinate system
b₀, B₁, And its homogeneous coordinates are b '₀, B '₁And Image I₀The viewpoint coordinates in three-dimensional space
O₀, Image I₁The viewpoint coordinates in the three-dimensional space of O₁Toss
You. At this time, the world coordinate system, that is, a general spatial coordinate
Reference system and viewpoint coordinate system O₀Is simple in the direction of each unit vector.
Just for the sake of simplicity. In addition, the viewpoint coordinates
System O₀And image I₀Z and y axes of the image coordinate system at
The directions are also parallel and coincide. The viewpoint coordinate system 1 is the viewpoint
T = O for coordinate system 0₁-O₀Translate only 3x3
It is assumed that the matrix R is rotated. At this time, this tertiary
The original vector T is called a base line vector. B and b₀, B₁Is related to λ₀, Λ₁Is a constant
, I is the identity matrix, and O is the image I₀Column vector at
Then, it becomes as follows. (Equation 5)₀b '₀  = [IO] B '... (1) (6)₁b '₁= [RT] B '... (2) From Equations (1) and (2), Equation (3) is derived using η as a constant.
You. (7)₁b '₁= Λ₀Rb '₀  + ΗT (3) Generally, world coordinates are transformed into viewpoint coordinates.
The matrix M is expressed by equation (4) in FIG.
It is called the division parameter. Equation (3) and each seat
From the three-dimensional positional relationship of the target, b '₁, T, Rb '₀Are the same
It is on a plane. This constraint is called epipolar constraint. Epi
Scalar triple product due to polar restraint | B '₁, T, Rb '₀| = 0 Holds. Due to the nature of the scalar triple product, (Equation 8) | B '₁, T, Rb '₀| = (B '₁, T_X(Rb '₀))                     = (| B '₁, ([T]_X× R) b '₀))                     = 0 (5) However, [T]_XIs a distortion symmetry that gives the same effect as the outer product of T
It is a matrix. Here, E = [T]_XR (Equation 9) (B '₁, Eb '₀) = B '₁ ^TEb '₀  = 0 (6) It is. E is called an essential matrix, and equation (6) is
It is generally called the epipolar equation. Next, a camera calibration matrix is defined. Image
Let the coordinates of the center be (x₀, Y₀), Focal length f, x direction, y
The coefficient of direction is k _x, K_yShear coefficient k_sAnd these
Camera Calibration Matrix Calibrated with Internal Camera Parameters
Is A, a_x= Fk_x  , A_y= Fk_y  , S = f
k_sIn a CCD camera having the same aspect ratio, a_x
= A_y  , X₀= 0, y₀  = 0, matrix A
Is a_x= A_y= A, defined by equation (7) in FIG.
It is. In the present invention, two captured images are given.
The actual value of the camera calibration matrix is unknown
Yes, you don't have to. Next, the image interpolation data calculation means 38B
Based on the input 8 corresponding points and using the above matrix
Find the foundation matrix F. The camera calibration matrix A defined earlier is
When used, the following relationship is established between screen coordinates and viewpoint coordinates:
Stand up. b '₀= A^-1p '₀, b '₁= A^-1p '₁ The basic matrix is expressed as F = (A^-1)^TEA^-1To be
From the definition of the essential matrix, ## EQU10 ## p '₁ ^TFp '₀= 0 (8) Becomes [Mathematical formula-see original document] p '₀= (P_0x, P_0y, P_0z…… (9) [Mathematical formula-see original document] p '₁= (P_1x, P_1y, P_1z…… (10) F = ｛F_{i, j}１ 1 ≦ i, j ≦ 3 And expanding equation (8), [Mathematical formula-see original document] p₀₁ f = 0 (11) Becomes Where p₀₁Is a 1 × 9 vector,
p '₀, P '₁Given that the homogeneous component of
And [Equation 14]   p₀₁  = (P_0xp_1x, P_0yp_1x, P_1x, P_0xp_1y, P_0yp_1y, P_1y, P_0x, p_0y, 1) ... (12) Can be saved. Let the i-th corresponding point vector be p_{01, i}
  And n rows and 9 columns in which n sets of corresponding point vectors are arranged
If the point matrix is P, P = ｛p_{01, i}；; 1 ≦ i ≦ n Becomes ## EQU15 ## Pf = 0 (13) Therefore, under the condition of {f} = 1 and f ≠ 0, the expression (1)
By solving 1), the fundamental matrix F can be obtained.
You. That is, here, the unknown F_{i, j}Is nine
However, normalization restricts the indefiniteness of the constant times.
Therefore, if more than 8 pairs of corresponding points are given, the linear solution
Apply and get a solution. That is, 2
Given eight corresponding points in one image,
Find the fundamental matrix F by solving eight simultaneous equations
(Step 606). In the calculation for obtaining the minimum eigenvector, free
Although the degree is set to 8, the rank of the fundamental matrix F is actually 2 and the matrix
Since the equation is always | F | = 0, the degree of freedom is 7. Follow
After satisfying this condition, singular value decomposition of F
Modify the number to be 2. D = ｛d_{i, j}｝; 1 ≦
i, j ≦ 3, U = ｛u_{i, j}｝; 1 ≦ i, j ≦ 3, V = ｛v
_{i, j}；; If 1 ≦ i, j ≦ 3, the singular value decomposition of F ## EQU16 ## F = UDV^T… (14) And Where D is a diagonal matrix and d₁₁≧ d_{twenty two}≧ d
₃₃And Then d₃₃The smallest value of
Finding the fundamental matrix ## EQU17 ## F = [d₁₁u_i1v_j1  + D_{twenty two}u_i2v_j2
 ]… (15) Becomes Next, the epipole shown in step 608 is
Define and calculate. Image I₁All points p 'on₁About
Thus, the image I for which the equation (8) is satisfied₀The upper point is e '₀, Painting
Statue I₀All points p 'on₀Equation (8) holds for
Una image I ₁The upper point is e '₁Then Fe '₀= 0, F^Te '₁  = 0 And e '₀, E '₁Is the epipole and the corresponding image
Represents a mapping of point locations. Next, the image is parallelized, and the image interpolation data is obtained.
The parallelized image of the basic image, which is the
To obtain feature line information and grid point information
And set the grid coordinates for the given image
(Step 612). First, FIG. 12 shows the positional relationship and description.
The two corresponding images are parallel and the corresponding points are parallel to the x-axis.
Mapping of images so that they are aligned on a straight line is called parallelization.
Say. First, to make the two images parallel, the image I_iTo
d_iThe rotation matrix for rotating θi about the axis is shown in FIG.
6) R (d_i, θ_i). In the following description,
I_iThe parallelized image is I ＾_i,Given the
Image I_iP above_i, The parallelized image I ＾_iThe point above
p ＾_iAnd Here, d_iIs parallel to the epipole
Unless you have a choice, you can
Vertical d_i= (-E_iy, E_ix, 0)^TAnd This rotation
As a result, Epipole e '_iMoves to the point at infinity, so R
(d_i, θ_i) e '_i Is 0. Therefore, (Equation 18) -e_ixd_iy sinθ_i+ e_iyd_ix sinθ_i + e_iz cosθ_i= 0 (17) θ_iAnd solve for [Equation 19] θ_i= (-1/2)-tan^-1((E_ixd_iy-E_iyd_ix) / E_iz… (18) Next, the rotation matrix around the z-axis that makes the epipolar line horizontal is
R (φ_i). e ''_i  = R (d_i, Θ_i) E^' _iWhen
Then, by rotation of R (φ), e ''_i  The y component of
Because (Equation 20) φ_i=-Tan^-1(E ''_iy/ E ''_ix… (19) p ＾_i= R (φ_i) R (d_i, θ_i) p_iConversion yields a constant
Except for the uncertainty, the corresponding points are aligned on the same scan line.
Huh. here, (Equation 21) H ＾_i= (R (φ_i) R (d_i, Θ_i))^-1 = R (d_i, ―Θ_i) R (-φ_i… (20) In other words, the corresponding point of the parallelized image p ＾₀, P ＾₁Is p ＾₀= H ＾^-1 ₀p '₀  … (21) p ＾₁= H ＾^-1 ₁p '₁    … (22) p ＾₀, p ＾₁Substituting into equation (8), (Equation 22) p '₁ ^TFp '₀= P ＾₁ ^TF ＾ p ＾₀… (23) F ＾ = H ＾^T ₁FH ＾₀Then, F ＾ is calculated as shown in FIG.
(24). Further, p ＾₀, P ＾₁  Y coordinates are equal
The 3 × 3 matrix that performs coordinate transformation so that₁Toki, picture
H is the homography of image 1.₁Then H₁= H ＾₁T₁ ^-1 T₁Is the equation (25) in FIG.₁ ^-1Is given by equation (26)
Given, p₀, P₁So that the y coordinates of
Ball, y₀= Y₁Is always given
 Homography for two images is given by
(Step 610). [Formula 23] H₀= R (d₀-θ₀) R (-φ₀)… (27) (Equation 24) H₁= R (d₁-θ₁) R (-φ₁) T₁ ^-1  … (28) Therefore, given image I₀To H₀ ^-1, Painting
Statue I₁To H₁ ^-1, The parallelized image I ＾₀,
I ＾₁Can be obtained. First, given image I₀, I₁The given
Divide the grid by the number of divisions, set a common grid coordinate system, and
Statue I₀G is the simultaneous coordinates of the above grid points₀i ', image I₁G on
₀g is the simultaneous coordinate of the grid point corresponding to i '₁i '(step
612). Next, the H obtained by the above-described method is used.₀ ^-1
To image I₀And H₁ ^-1To image I₁By applying to
Line image I ＾₀, I ＾₁(Step 614). Further, the parallelized image I ＾₀G on₀against i '
The coordinates of the corresponding grid point are g₀i ＾, parallelized image I ＾₁G on
₁Let g be the coordinates of the grid point corresponding to i '₁i ＾. This split
Number is picture I₀, I₁, I ＾₀, I ＾₁Common to the image
(Step 61)
6). Next, the obtained grid points are represented by H₀ ^-1, H₁ ^-1By
To obtain grid points on the parallelized image. Further features
For the start and end points of the line,₀ ^-1, H₁ ^-1Convert by
Then, a point on each parallelized image is obtained. Characteristic line
For any point other than ₀ ^-1, H₁ ^-1By each
Is converted to a parallel image I ＾₀, I ＾₁Ask for. (S
Steps 614, 616) Next, a method for obtaining an interpolated image of the viewpoint s will be described.
State. Image morphing the obtained parallelized image,
A parallelized interpolation image I ＾ s of a desired viewpoint s is obtained. However
Then, the viewpoint s becomes the internal division ratio s (0 ≦ s ≦
Let it be given in 1). First, the parallelized image I ＾₀, I ＾₁Up
Grid point g₀i ＾, g₁Linear interpolation of i ＾ by internal division ratio s
Then g_sLet i ＾ be the coordinates of the grid point of the parallelized interpolated image I ＾ s
Is obtained (step 700). Next, the parallelized image I ＾₀, I ＾₁Convert up
From the coordinates of the start point and end point of the feature line
The coordinates of the corresponding start and end points on the parallelized interpolated image of viewpoint s
The value is obtained and set as a characteristic line on the parallelized interpolation image I ＾ s. flat
In the lined image, the corresponding points are on the same scan line.
Linear with the internal division ratio s only for the x-coordinate value
Can be obtained by interpolation to
The point p₀＾ i and p₁＾ i (0 ≦ i ≦ n, n endpoints of the feature line
), The corresponding point p for the desired viewpoint s_s＾ i
 Is given by the following equation. (Equation 25) p_s＾ i = (1-s) p₀＾ i + sp₁＾ i… (29) [0086] Any other than the characteristic line on the parallelized interpolation image I ＾ s
The important point is the difference between the obtained feature line on the parallelized interpolation image I ＾ s.
Due to the relationship, the parallelized image I ＾₀, I ＾₁Picture of morphin
Sought. The specific method of image morphing is well known.
Are not described in detail here (step 7).
02). Next, the parallelization complement obtained in this way is calculated.
A procedure for converting the interim image I ＾ s into the interpolated image Is will be described.
(Step 704). FIG. 15 shows the positional relationship. First,
An arbitrary point on the obtained parallelized interpolation image I ＾ s is represented by p_s＾
Then, p_s座標 coordinate value and grid point g_si ＾ (0 ≦ i ≦ n;
n is the number of grid points), and_s＾ is the parallelization complement
Which grid region of the inter-image I ＾ s is included is determined. lattice
Point g_sLet Rg be the grid area with i ＾ as the reference point._sgrid represented by i ＾
G the local coordinates of the region_sWhen i ＾ (u, v), p_s＾
Grid point g_sGrid region Rg with k ＾ as reference point_sincluded in k ＾
And At this time, p_s＾ with local coordinates g_sk ＾ (u,
v). Next, the given image I₀, I₁Territory similar to
Image I₀, I₁A grid coordinate system similar to the one set in
And set the grid point of the interpolation image Is to g_si 'and the parallelized image
Same as setting local coordinate system at grid points on image
The grid point g_sLet g be the local coordinates with i 'as the reference point_si '(u,
v), and the grid region is represented by Rg_sRepresented by i '. Interpolated image I_sAbove, the parallelized image I_s点 Point on
p_sPoint p corresponding to ＾_s'Is p_sGrid region containing ＾
Rg_sgrid region Rg on the interpolated image corresponding to k ＾_sk '
Rare, local coordinates g_sk ＾ (u, v) is the local coordinate g_sk '(u,
v) and the point p_s'Position. Less than
By repeating the above procedure, all the parallelized images I
_s点 Point p on_sAbout ＾, the image I want_sFind the above point
You. Interpolated image I of final viewpoint s_sIs obtained. That is, the image I₀And image I₁Ask from
A homograph of the intermediate image Is (0 ≦ s ≦ 1) being sought
H_sAnd At this time, the homography conversion H_sAlready
It is good to know, but in fact, unknown. Intermediate image
Homography H to find out_sIs θ_i, Φ_i, D
_i, T₁Can be obtained by direct linear interpolation
However, in general, they often do not retain the shape of the object in the image.
Therefore, an image was obtained by the above method. Although the outline of the above procedure has been merely described,
For more information, see a method called viewpoint morphing.
Is. It should be noted that the image is browsed and viewed in the image browsing method.
What is the image that changes the point if it fits in a photo (video)
Is also good. That is, the following are conceivable. # 1 General objects (objects, creatures (humans, animals (including insects)
Mu), flowers, trees) # 2 Buildings (built houses for sale, condominiums, public facilities)
In) # 3 Landscape, scenery (introducing travel pamphlets and historical landscapes) # 4 Interior example # 5 Examples of services (cleaning, nursing care, babysitting
ー, etc) Other than this, what can be included in video from multiple viewpoints
If so, change the viewpoint in this image browsing method and browse
can do. Further, in this embodiment, corresponding points and features
The line shows an example of inputting with a mouse or the like.
Apply corresponding software to extract corresponding points and feature lines
Then, it may be an input value. The program of this image browsing method is
Browser software, which is a browsing software for
In addition to being used as an air plug-in,
It is also used as viewing software. Further, the rendering of the parallelized interpolation image to the interpolation image is performed.
The calculation method may be an interpolation calculation method other than the above. In this embodiment, a given image is
And common grid coordinates are set, but common grid coordinates are set.
If it is not possible to determine
Thus, the corresponding grid points are linearly interpolated from the viewpoint s.
Thus, the lattice point of the desired viewpoint s may be set.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a connection configuration diagram showing an entire system according to a specific embodiment of the present invention. FIG. 2 is a block diagram illustrating the configuration of an embodiment of the present invention (the invention of claim 2). FIG. 3 is a block diagram illustrating a configuration of an embodiment of another invention (the invention of claim 3). FIG. 4 is a flowchart (1) showing a processing procedure of a web server. FIG. 5 is a flowchart showing a processing procedure of the web server (part 2). FIG. 6 is a flowchart showing a processing procedure of a user's computer. FIG. 7 is a flowchart (part 2) showing a processing procedure of a web server of another invention. FIG. 8 is a flowchart showing a processing procedure of a computer according to another invention. FIG. 9 is a flowchart showing a procedure for creating basic data. FIG. 10 is a flowchart illustrating a procedure of an image interpolation process. FIG. 11 is an explanatory diagram of a positional relationship between two given images. FIG. 12 is an explanatory diagram of parallelization of two given images. FIG. 13 is a positional relationship between a parallelized interpolated image and an interpolated image. FIG. 14 is an explanatory diagram of a determinant used in the embodiment. FIG. 15 is an explanatory diagram of a determinant used in the embodiment. [Description of References] 5, 6 Local area networks 7a, 7b Internet network 8 Public line 9 Management terminals 10a, 10b, 10c User computer 20 Web servers 70a, 70b Client computers 30, 50 ... Control devices 40, 60 ... Input devices 41, 61 ... External storage devices 42, 62 ... Display devices 31, 56 ... Interpolation image generation means 32A ... Basic data storage means 32B ... Image interpolation data calculation means 33A ... Basic image input means 33B ... Corresponding information input means 34, 53 ... receiving means 35 ... command receiving means 36, 51 ... transmitting means 37a ... image data transmitting means 37b ... basic data transmitting means 38 ... homepage publishing means 39, 55 ... connecting means 52 ... command transmitting means 54a ... Image data receiving means 54b ... Basic data receiving means

Claims (1)

Claims: 1. A user's computer and a web server that publishes a homepage on a network are connected via a network, and a user can access the homepage on the homepage via the user computer and the network. When browsing image data that is an image of an object, according to a command from the user computer, the image data of the object from an arbitrary viewpoint desired by the user is displayed, and the image data from the commanded viewpoint is Using the basic data stored in the web server, the image of the object is imaged from a plurality of viewpoints in advance and predetermined preprocessing is performed,
A homepage image browsing method obtained by an interpolation image generation method as a predetermined interpolation operation. 2. The basic data includes image interpolation data for performing the interpolation operation and a plurality of basic image data, wherein the image interpolation data includes a plurality of corresponding points in the basic image in the plurality of basic images. 2. The method according to claim 1, wherein the method is performed by calculating a predetermined feature amount. 3. A plurality of corresponding points in the basic image are at least eight sets of corresponding point information, and the predetermined feature amount is a plurality of corresponding feature line information in the basic image. The method for browsing homepage images according to claim 2, wherein 4. The apparatus according to claim 3, wherein a plurality of corresponding points in the basic image or a plurality of feature line information in the plurality of basic images are input and set for the plurality of basic images. How to browse homepage images. 5. The image interpolation data obtained from the eight or more sets of corresponding point information indicates that when an epipolar equation is established between two given basic images, A is 3 ×
3, a camera calibration matrix, A ^-1 is an inverse matrix of A, (A ^-1 ) ^T is a transpose of (A ^-1 ), E is a 3 × 3 essential matrix, and p ′ ₀ and p ′ ₁ are images. Let I _{0 be} the homogeneous coordinates of the corresponding point in each image coordinate system of the image I ₁ , and if F = (A ⁻¹ ) ^T EA ⁻¹ , then (1) p ′ ₁ ^T Fp ′ ₀ = 0 (2) Therefore, a 3 × 3 fundamental matrix F obtained by solving the simultaneous equations obtained by expanding the equation (2) with the given information of eight or more sets of corresponding points. 5. A parallelized image obtained by H ₀ ^-1 and H ₁ ^-1 which are inverse matrices of homography H ₀ and H ₁ which are transformation matrices determined by using Homepage image browsing method. 6. The method for generating an interpolated image, comprising: obtaining an interpolation point of a viewpoint corresponding to a command in the parallelized image to obtain a parallelized interpolated image in the parallelized image;
The homepage image browsing method according to claim 5, wherein a predetermined operation corresponding to homography conversion of a corresponding viewpoint is performed on the obtained parallelized interpolation image to obtain an interpolation image of a desired viewpoint. . 7. The web server receives a command from the user computer and uses the stored basic data to generate image data from an arbitrary viewpoint desired by the user by the interpolation image generation method. The homepage image browsing method according to claim 6, wherein the method is generated and transmitted to the user computer. 8. The web server receives a command from the user computer, transmits the stored basic data to the user computer, the user computer receives the basic data from the web server, 7. The homepage image browsing method according to claim 6, wherein the interpolation image generating method generates image data from an arbitrary viewpoint desired by the user. 9. The web server used in the homepage image browsing method according to claim 7, wherein: a homepage publishing unit that publishes the homepage; a connection unit that connects to the user computer so as to be accessible via a network; Command receiving means for receiving a command from the user computer, basic image input means for inputting the basic image, a plurality of corresponding points in the basic image, or corresponding information input means for inputting a predetermined feature amount, Using the input basic image, image interpolation data calculation means for calculating the image interpolation data from the correspondence information, basic data storage means for storing the basic data, and performing the reception using the stored basic data. Image that generates image data from an arbitrary viewpoint according to a command from a user computer Web server and having a forming means, and an image data transmitting means for transmitting the image data to which the generated in the user computer. 10. The user computer used in the homepage image browsing method according to claim 7, wherein: a connection means for connecting to the web server so as to be accessible via a network; and a command from the user computer to the web server. A user computer comprising: a command transmitting unit that transmits; and an image data receiving unit that receives image data from an arbitrary viewpoint desired by the user transmitted from the web server. 11. The web server used in the homepage image browsing method according to claim 8, wherein: a homepage publishing unit that publishes the homepage; a connection unit that connects to the user computer so as to be accessible via a network; Basic image input means for inputting the basic image, corresponding information input means for inputting a plurality of corresponding points in the basic image, or a plurality of feature line information; and Image interpolation data calculation means for calculating image interpolation data; basic data storage means for storing the basic data; command reception means for receiving a command from the user computer; and the stored basic data to the user computer. And a basic data transmitting means for transmitting the data to the web server. 12. The user computer used in the homepage image browsing method according to claim 8, wherein: a connection means for connecting to the web server so as to be accessible via a network; and a command from the user computer to the web server. Command transmitting means for transmitting; basic data receiving means for receiving basic data transmitted from the web server; andinterpolated image generating means for generating image data from an arbitrary viewpoint using the received basic data; A user computer, comprising: 13. The web server used in the homepage image browsing method according to claim 7, wherein: a homepage publishing procedure for publishing the homepage; a connection procedure for making an accessible connection to the user computer via a network; A command receiving procedure for receiving a command from the user computer, a basic image input procedure for inputting the basic image, a plurality of corresponding points in the basic image, or a corresponding information input procedure for inputting a plurality of feature line information, The input basic image, an image interpolation data calculation procedure for calculating the image interpolation data from the correspondence information, a basic data storage procedure for storing the basic data, and using the stored basic data, Interpolation to generate image data from any viewpoint according to command from user computer An image generation procedure, a recording medium the image data the generated storing the web server readable program characterized by having an image data transmission step of transmitting to the user computer. 14. The user computer used in the homepage image browsing method according to claim 7, wherein: a connection procedure for making an access to the web server via a network; and a command from the user computer to the web server. Storing the program readable by the user computer, comprising: a command transmission procedure for transmitting; and an image data receiving procedure for receiving image data from an arbitrary viewpoint desired by the user transmitted from the web server. Recording medium. 15. The web server used in the homepage image browsing method according to claim 8, wherein a homepage publishing procedure for publishing the homepage, a connection procedure for making an accessible connection with the user computer via a network, A basic image inputting step of inputting the basic image, a corresponding information inputting step of inputting a plurality of corresponding points in the basic image, or a plurality of feature line information, the input basic image, and An image interpolation data calculation procedure for calculating image interpolation data, a basic data storage procedure for storing the basic data, a command receiving procedure for receiving a command from the user computer, and the stored basic data for the user computer. The web server having a basic data transmission procedure for transmitting data to the web server Recording medium which stores only take possible program. 16. The user computer used in the homepage image browsing method according to claim 8, wherein: a connection procedure for making an accessible connection to the web server via a network; and a command from the user computer to the web server. A command transmission procedure for transmitting, a basic data receiving procedure for receiving basic data transmitted from the web server, and an interpolation image generating procedure for generating image data from an arbitrary viewpoint using the received basic data, A recording medium storing the program readable by the user computer.