KR101604002B1 - Method, device and non-trnasitory computer-readable recording media for supporting relay broadcasting using mobile terminal - Google Patents

Abstract

According to one embodiment of the present invention, a method for supporting relay broadcasting using a mobile terminal may comprise: (a) receiving at least one of a first video data and a first audio data from a first photographer terminal device, and receiving at least one of a second video data and a second audio data from a second photographer terminal device; (b) generating metadata for edition of relay broadcasting contents including at least one of the first video data, the second video data, the first audio data and the second audio data, based on edition input from a user; and (c) transmitting data to be included in the relay broadcasting contents among the first video data, the second video data, the first audio data and the second audio data, and metadata for the edition, to a viewer terminal device, wherein data to be included in the relay broadcasting contents among the first video data, the second video data, the first audio data and the second audio data being transmitted to the viewer terminal device, in status of raw data received from the first photographer terminal device or the second photographer terminal device.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method, apparatus, and non-transitory computer readable recording medium for supporting a relay broadcast using a mobile terminal apparatus. 2. Description of the Related Art [0002]

The present invention relates to a method, apparatus and non-transitory computer readable recording medium for supporting relay broadcasting using a mobile terminal apparatus.

Basically, since a lot of hardware resources (e.g., CPU, memory, etc.) are required to process video signals and audio signals, a video signal and an audio signal It is required to have expensive equipment having a very high specification in order to produce a relay broadcast signal by processing and editing in real time. As the broadcasting and communication technology has been developed recently, the performance of the apparatus has been increased and popularized. However, since the video signals and audio signals to be processed by these apparatuses have also increased in resolution and quality, It is difficult to process and edit a plurality of video signals and audio signals.

In particular, although a general user can generate his or her own video or audio using a camera or a microphone provided in a mobile terminal device such as a smart phone or a tablet, due to the limitation of the performance of the mobile terminal device, The generated video signal or audio signal can be processed only to the extent of simply uploading or downloading, and it is difficult to produce a relay broadcast content in which a plurality of video signals and audio signals are combined. Accordingly, But it has limitations in utilizing video and audio in various forms.

It is an object of the present invention to solve all the problems described above.

Further, the present invention is characterized by receiving first video data or first audio data from the first photographer terminal device, receiving second video data or second audio data from the second photographer terminal device, And generates metadata related to editing of relay broadcast contents including at least one of the first video data, the second video data, the first audio data, and the second audio data, The video data or the audio data included in the above relay broadcast contents are transmitted to the viewer terminal device via the first camera terminal device or the second camera terminal device, to be transmitted to the viewer terminal device while maintaining the raw data state Written and that many computational resources to create even relay broadcast contents unless the processing is required with respect to the video data or audio data received from a plurality of mobile terminal devices for other purposes.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to an aspect of the present invention, there is provided a method for supporting relay broadcasting using a mobile terminal apparatus, the method comprising: (a) receiving at least one of first video data and first audio data from a first photographer terminal device, Receiving at least one of second video data and second audio data from a terminal device, (b) receiving the first video data, the second video data, the first audio data, and the second audio data with reference to an editing input from a user, Generating metadata related to editing of relay broadcast contents including at least one of the first audio data, the second audio data, and the second audio data; and (c) 2 data included in the relay broadcast content and the metadata relating to the edit are transmitted to the viewer terminal device Wherein the first video data, the second video data, the first audio data, and the second audio data are included in the relay broadcast contents, And transmitting to the viewer terminal device while maintaining the raw data state received from the device.

According to another aspect of the present invention, there is provided an apparatus for supporting relay broadcasting using a mobile terminal apparatus, the apparatus comprising: a receiving unit receiving at least one of first video data and first audio data from a first photographer terminal apparatus, A second video data, a second audio data, and a second audio data with reference to an editing input from a user, A relay broadcast content editor for generating metadata relating to editing of relay broadcast contents including at least one of the first video data, the second video data, the first audio data, and the second audio data; The data included in the relay broadcast content and the metadata A relay broadcast content transmitting unit for transmitting the relay broadcast content to the viewer terminal apparatus; and a relay broadcast content transmitting unit for transmitting the relay broadcast content to the viewer terminal apparatus, the relay broadcast content transmitting unit transmitting the relay broadcast content including the first video data, the second video data, the first audio data, And the second camera terminal device is transmitted to the viewer terminal device while maintaining the raw data state received from the device or the second photographer terminal device.

In addition, there is further provided a computer readable recording medium for recording a computer program for executing the method and an apparatus and an apparatus for implementing the present invention.

According to the present invention, since it is possible to produce the relay broadcast contents without the processing requiring a lot of calculation resources for the video data or the audio data, the relay broadcast contents can be produced by using a conventional wireless communication network (for example, a mobile communication network such as 3G or LTE, A general user having only a mobile terminal device capable of connecting to a local area network such as Wi-Fi, Wi-Fi Direct and LTE Direct can easily produce relay broadcast contents. .

In addition, according to the present invention, various editing effects can be achieved while the user is not significantly affected by the performance of the apparatus.

In addition, according to the present invention, an advantage that a user can easily obtain information (for example, synchronized time information) which is difficult to obtain in a conventional relay broadcasting equipment from the mobile terminal device and can be utilized in producing relay broadcast contents .

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a configuration of an overall system for implementing the present invention. FIG.
2 is a diagram illustrating an internal configuration of an intermediary terminal device according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating a configuration for generating relay broadcast contents according to an exemplary embodiment of the present invention. Referring to FIG.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and attributes described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

Configuration of the entire system

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a configuration of an overall system for implementing the present invention. FIG.

1, an overall system according to an exemplary embodiment of the present invention may include a communication network 100, an intermediary terminal device 200, a photographer terminal device 300, and a viewer terminal device 400. As shown in FIG.

First, the communication network 100 according to an embodiment of the present invention can be configured without regard to communication modes such as wired communication and wireless communication. The communication network 100 can be a local area network (LAN), a metropolitan area network ), A wide area network (WAN), and the like. Preferably, the communication network 100 referred to herein includes well-known short range wireless communication networks such as Wi-Fi, Wi-Fi Direct, LTE Direct, Bluetooth, . However, the communication network 100 may include, at least in part, a known wire / wireless data communication network, a known telephone network, or a known wire / wireless television communication network, without being limited thereto.

Next, the relay terminal device 200 according to the embodiment of the present invention receives the first video data or the first audio data from the first photographer terminal device 300A and receives the first video data or the first audio data from the second photographer terminal device 300B 2 video data or second audio data, and refers to an editing input from a user to reproduce the broadcast content including at least one of the first video data, the second video data, the first audio data, and the second audio data And transmits the video data or the audio data included in the above relay broadcast contents and the above-described edit metadata to the viewer terminal device. The video data or audio data included in the above relay broadcast contents The audio data is the raw data r (r) received from the first photographer terminal device 300A or the second photographer terminal device 300B aw data to be transmitted to the viewer terminal apparatuses so that the relay broadcast contents can be produced without processing the video data or the audio data received from the plurality of mobile terminal apparatuses requiring a lot of operation resources Function can be performed. Any digital device having memory means such as a smart phone, a tablet, a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, etc., 200).

The configuration and function of the relay terminal device 200 according to the present invention will be described in detail with reference to the following detailed description.

Next, according to one embodiment of the present invention, the photographer terminal device 300 is a digital device including a function of transmitting video data photographed by a photographer or recorded audio data to the relay terminal device 200, The camera terminal device 300 according to the present invention can be used as any digital device having memory means such as a tablet, a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, As shown in FIG.

Next, in accordance with an embodiment of the present invention, the viewer terminal 400 receives the video data, the audio data, and the metadata included in the relay broadcast contents transmitted from the relay terminal device 200, A digital device having a memory means such as a smart phone, a tablet, a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, and the like, Any device can be adopted as the viewer's apparatus 400 according to the present invention.

In particular, the relay terminal device 200, the photographer terminal device 400, and the viewer terminal device 400 may include a dedicated application (not shown) for supporting the production of relay broadcast contents. Such an application may be downloaded from an external service providing server (not shown).

Although the intrinsic functions that can be performed by the relay terminal device 200, the photographer terminal device 300 and the viewer terminal device 400 are mainly described in the present specification, the relay terminal device 200, It should be noted that the terminal device 300 and the viewer terminal device 400 do not necessarily perform their own functions. That is, the same mobile terminal device can be any of the relay terminal device 200, the photographer terminal device 300, and the viewer terminal device 400 depending on what purpose it is used for. For example, when a user photographs or records using his / her mobile terminal device, the corresponding mobile terminal device can be the photographer terminal device 300, and when the user performs relay using his / her mobile terminal device The mobile terminal device may be an intermediary terminal device 200. When the user uses the mobile terminal device of his or her own to watch the mobile terminal device, the mobile terminal device may be the viewer terminal device 400. [

Configuration of relay terminal device

Hereinafter, an internal configuration of the intermediary terminal device 200 that performs an important function for the implementation of the present invention and functions of the respective components will be described.

2 is a diagram illustrating an exemplary internal configuration of an intermediary terminal device 200 according to an embodiment of the present invention.

2, an intermediary terminal apparatus 200 according to an exemplary embodiment of the present invention includes a data receiving unit 210, a relay broadcast content editor 220, a relay broadcast content transmitter 230, a communication unit 240, And may include a controller 250. According to an embodiment of the present invention, at least a part of the data receiving unit 210, the relay broadcast content editing unit 220, the relay broadcast content transmitting unit 230, the communication unit 240, Lt; / RTI > may be program modules in communication with the mobile station 200. These program modules may be included in the intermediary terminal device 200 in the form of an operating system, an application program module, and other program modules, and may be physically stored on various known memory devices. Such program modules may also be stored in a remote storage device capable of communicating with the relay terminal device 200. [ These program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types as described below in accordance with the present invention.

The data receiving unit 210 may receive video data or audio data from at least one photographer terminal 300 capable of communicating with the intermediary terminal 200. In this case, More specifically, according to one embodiment of the present invention, the first video data or the first audio data is received from the first photographer terminal device 300A, and the second video data or the second video data from the second photographer terminal device 300B And can perform a function of receiving audio data. According to an embodiment of the present invention, the video data or audio data received from the photographer terminal device 300 may be separately encoded and decoded for editing or relaying in the initial state in which the corresponding data is generated, , And a raw data format in which processing such as compression is not performed.

Also, according to an embodiment of the present invention, the video data received from the photographer terminal device may be composed of a plurality of images containing a still image, and the plurality of images may have a frame rate of at least 1 frame per second ) Can constitute video data.

Next, in accordance with an embodiment of the present invention, the relay broadcast content editing unit 220 edits relay broadcast contents including data received from the first photographer terminal device 300A and the second photographer terminal device 300B Function can be performed.

In more detail, according to an embodiment of the present invention, the relay broadcast content editing unit 220 extracts the first video data, the second video data, the first audio data And generating second meta data relating to editing of a relay broadcast content including at least one of the first audio data and the second audio data. For example, the meta data may include at least one of the first video data and the second video data, at least one of the first audio data and the second audio data, information about switching between the first video data and the second video data, Information on the caption, and the like.

Also, according to an embodiment of the present invention, the metadata related to the editing of the relay broadcast contents may include text-based data having a format such as XML or JSON.

FIG. 3 is a diagram illustrating a configuration for generating relay broadcast contents according to an exemplary embodiment of the present invention. Referring to FIG.

3, an intermediary terminal device 200 receives first video data and first video data from a first photographer terminal device 300A through a protocol such as RTSP (Real Time Streaming Protocol) and HTTP (Hyper Text Transfer Protocol) 2 audio data, and can receive the second video data and the second audio data from the second photographer terminal device 300B. Next, the transponder confirms the first video data, the second audio data, the second video data, and the second audio data reproduced on the intermediary terminal device 200, while editing the relay broadcast contents including at least a part of these data , And the information on the above editing can be stored as metadata. For example, the relay can reproduce the effect that the first video data and the second audio data are reproduced together to generate a black and white effect and display a subtitle "Carryculture ". Next, the intermediary terminal device 200 may package the video data or the audio data included in the relay broadcast contents and the metadata related to editing in a predetermined container 310 and transmit the same to the viewer terminal device 400 .

In addition, according to an embodiment of the present invention, the relay broadcast content editing unit 220 may include a synchronized time acquisition unit that can acquire a time code serving as a reference for editing the relay broadcast content from the photographer terminal 300 Can be set based on information. In particular, according to an embodiment of the present invention, a time code serving as a criterion for editing the relay broadcast contents is acquired from the photographer terminal device which transmitted the latest received data among the video data and audio data included in the relay broadcast contents Lt; RTI ID = 0.0 > synchronized < / RTI > Since the plurality of photographer terminal devices 300A and 300B that transmit video data or audio data in real time to the relay terminal device 200 according to the present invention already have synchronized time information through the mobile communication network or the like, By utilizing the synchronized time information obtained from any one of the apparatuses 300A and 300B, it is possible to easily set a time code between video data and audio data without performing a separate synchronization operation, The content can be generated.

In addition, according to an embodiment of the present invention, the relay broadcast content editing unit 220 receives audio data from only one of the plurality of photographer terminal devices 300A and 300B and receives audio data from the remaining photographer terminal devices It is possible to receive only the video data without receiving the video data and to include only the audio data received from the photographer terminal device in the relay broadcast contents. Therefore, according to the present invention, it is possible to save network resources required for receiving a plurality of audio data from each of a plurality of photographer terminal devices, and to combine a plurality of audio data in producing relay broadcast contents It is possible to save the operation resources required.

Next, in accordance with an embodiment of the present invention, the relay broadcast content transmitting unit 230 transmits the first video data, the second video data, the first audio data, and the second audio data received from the plurality of photographer terminal devices 300A and 300B 2 to the viewer terminal device 400. The viewer terminal device 400 may be configured to transmit the metadata included in the relay broadcast content and the metadata related to the editing. Specifically, according to an embodiment of the present invention, the first video data, the second video data, the first audio data, or the second audio data included in the relay broadcast contents are transmitted to the first photographer terminal device 300A or the second mobile Can be transmitted to the viewer terminal device 400 while maintaining the raw data state received from the terminal device 300B.

Meanwhile, in accordance with an embodiment of the present invention, the viewer terminal device 400 receiving the video data or audio data included in the relay broadcast content and the metadata related to the edit transmits the relay broadcast content using the above- The video data or the audio data can be reproduced with reference to the metadata related to editing, thereby enabling the relay broadcast contents intended by the relay terminal 200 to be provided to the viewer.

According to an embodiment of the present invention, the communication unit 240 performs a function of allowing the intermediary terminal device 200 to communicate with an external system such as the photographer terminal device 300 or the viewer terminal device 400 .

The control unit 250 controls the flow of data between the data receiving unit 210, the relay broadcast content editing unit 220, the relay broadcast content transmitting unit 230, and the communication unit 240. [ . That is, the control unit 250 controls the flow of data between the components of the relay terminal device 200 to / from the outside or the outside, thereby controlling the data receiving unit 210, the relay broadcast content editing unit 220, And controls the transmission unit 230 and the communication unit 240 to perform their own functions.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

100: Network
200: repeater terminal device
210: Data receiving unit
220: Broadcast broadcast content editor
230: Relay broadcast content transmission unit
240:
250:
300: photographer terminal device
300A: first photographer terminal device
300B: second photographer terminal device
400: viewer terminal device

Claims (10)

A method for supporting relay broadcasting using a mobile terminal apparatus,
(a) receiving at least one of first video data and first audio data from a first photographer terminal device, receiving at least one of second video data and second audio data from a second photographer terminal device,
(b) a metadata relating to editing of relay broadcast contents including at least one of the first video data, the second video data, the first audio data, and the second audio data with reference to an editing input from the user; Generating data; and
(c) transmitting, from the first video data, the second video data, the first audio data, and the second audio data, the data included in the relay broadcast content and the metadata related to the edit to the viewer terminal device - the data included in the relay broadcast content among the first video data, the second video data, the first audio data, and the second audio data is received from the first photographer terminal device or the second photographer terminal device And is transmitted to the viewer terminal device while maintaining a raw data state.
Lt; / RTI >
Wherein the time code serving as a reference for editing the relay broadcast contents is based on already synchronized time information obtained from the photographer terminal device which transmitted the latest received data among the data included in the relay broadcast contents Is set,
Wherein the already synchronized time information is determined in a mobile communication network to which a photographer terminal device that has transmitted the latest received data among data included in the relay broadcast contents belongs,
In the transmitting step,
The method comprising the steps of: packaging data included in the relay broadcast content among the first video data, the second video data, the first audio data, and the second audio data, and the metadata relating to the editing in a predetermined container, And transmitting the packaged container to the viewer terminal device. The method according to claim 1,
Wherein the metadata related to the editing includes information about switching between the first video data and the second video data, at least one of the first video data and the second video data, the first audio data, Information about a combination between at least one of audio data and audio data, and information about a subtitle. delete delete The method according to claim 1,
In the step (a)
Receiving only one of the first audio data and the second audio data,
In the step (b)
And wherein only the received one of the first audio data and the second audio data is included in the relay broadcast content. The method according to claim 1,
In the step (a)
At least one of the first video data, the first audio data, the second video data, and the second audio data is Wi-Fi, Wi-Fi Direct, or LTE Direct Wherein the method is received via at least one local area wireless technology. The method according to claim 1,
Wherein the video data comprises a plurality of images containing still images. The method according to claim 1,
Wherein the metadata includes text-based data. 9. A non-transitory computer program for performing the method according to any one of claims 1, 2 and 5 to 8. An apparatus for supporting relay broadcasting using a mobile terminal apparatus,
A data receiving unit that receives at least one of first video data and first audio data from the first photographer terminal device and receives at least one of second video data and second audio data from the second photographer terminal device,
Generates metadata relating to editing of relay broadcast contents including at least one of the first video data, the second video data, the first audio data, and the second audio data with reference to an editing input from the user A relay broadcast content editing unit, and
A relay broadcast content transmission unit that transmits data included in the relay broadcast content among the first video data, the second video data, the first audio data, and the second audio data, and the metadata related to the edit to the viewer terminal unit The data included in the relay broadcast content among the first video data, the second video data, the first audio data, and the second audio data is received from the first photographer terminal device or the second photographer terminal device Is transmitted to the viewer terminal device while maintaining the raw data state -
Lt; / RTI >
Wherein the time code serving as a reference for editing the relay broadcast contents is based on already synchronized time information obtained from the photographer terminal device which transmitted the latest received data among the data included in the relay broadcast contents Is set,
Wherein the already synchronized time information is determined in a mobile communication network to which a photographer terminal device that has transmitted the latest received data among data included in the relay broadcast contents belongs,
Wherein the relay broadcast content transmitting unit transmits the data included in the relay broadcast content among the first video data, the second video data, the first audio data, and the second audio data, and transmits the packaged container to the viewer terminal device.

Images