JP7240134B2 - Information display device, electronic pen, display control method, and display control program - Google Patents

Description

The present invention relates to an information display device, an electronic pen, a display control method, and a display control program.

2. Description of the Related Art Conventionally, an information display device having a touch panel for accepting handwritten input has been widely used. In such an information display device, a touch panel receives a handwritten input with a pen or a finger as an indicator, and displays a handwritten trajectory representing a character or a figure (see, for example, Patent Document 1).

JP 2017-152018 A

However, when changing the display mode of the handwritten trajectory in the conventional information display device, it is necessary for the user to perform an input operation for changing the display mode of the handwritten trajectory with an indicator before or after handwriting input. There is a problem that the operation is complicated.

Further, when the information display device displays a GUI (window, icon, button, etc.) for receiving an input operation for changing the display mode, such a GUI can be used to display a part of the limited display area of the information display device. are occupied, and the operability of handwriting input may be further deteriorated.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an information display device, an electronic pen, a display control method, and a display control program that improve operability when changing the display mode of a handwritten trajectory.

An information display device according to a first aspect includes a touch panel that accepts handwriting input by an indicator, a display unit that displays the locus of handwriting accepted by the touch panel, a voice input unit that accepts voice input including a voice command, and handwriting. and a controller for controlling a display mode of the handwritten trajectory on the display unit according to a voice command received by the voice input unit in an input state.

The electronic pen according to the second aspect functions as the indicator.

A display control method according to a third aspect is a method used in an information display device having a touch panel that accepts handwritten input using an indicator. The display control method includes, in a handwriting input state, a voice input step of receiving a voice input including a voice command, a display step of displaying a trace of handwriting received by the touch panel, and a voice command received in the voice input step. and a control step of controlling a display mode of the handwritten trajectory in the display step according to the above.

A program according to a fourth aspect provides an information display device having a touch panel that accepts handwriting input by an indicator, in a handwriting input state, a voice input step of accepting voice input including a voice command; and a control step of controlling the display mode of the handwritten trajectory in the display step according to the voice command received in the voice input step.

According to one aspect of the present invention, it is possible to provide an information display device, an electronic pen, a display control method, and a display control program that improve operability when changing the display mode of a handwritten locus.

1 is an external view of an information display device according to an embodiment; FIG. 1 is a block diagram showing the functional configuration of an information display device according to one embodiment; FIG. 1 is an external view of an electronic pen according to one embodiment; FIG. 1 is a block diagram showing the functional configuration of an electronic pen according to one embodiment; FIG. It is a figure which shows an example of the handwriting input screen which concerns on one Embodiment. FIG. 11 is a diagram showing an example of changing the display mode of a handwritten trajectory in stages according to one embodiment; FIG. 11 is a diagram showing an example of changing the display mode of a handwritten trajectory according to one embodiment; FIG. 11 is a diagram showing another example of changing the display mode of the handwritten trajectory according to one embodiment; It is a figure which shows the handwriting input state which concerns on one Embodiment. It is a figure which shows an example of the operation|movement flow of the information display apparatus which concerns on one Embodiment, and an electronic pen. FIG. 7 is a diagram showing another example of the operation flow of the information display device and the electronic pen according to one embodiment;

Embodiments will be described with reference to the drawings. In the description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

(Configuration of information display device)
The information display device according to one embodiment can be a terminal such as a smartphone terminal or a tablet terminal, for example. However, the information display device is not limited to such a terminal, and may be, for example, a personal computer, an electronic blackboard, or an in-vehicle information display device.

FIG. 1 is an external view of an information display device 100 according to one embodiment.

As shown in FIG. 1, the information display device 100 has a touch screen display 110, a microphone 120, a speaker 130, and a camera 140. As shown in FIG.

The touch screen display 110 is provided with its display surface exposed from the housing 101 of the information display device 100 . The touch screen display 110 has a touch panel 111 and a display section (display) 112 .

Touch panel 111 receives an operation input (touch input) to information display device 100 . The touch panel 111 detects a touch of a user's finger, an electronic pen, or the like as an indicator. Methods for detecting a touch include, for example, a resistive film method and a capacitive method, but any method may be used. The touch panel 111 detects a user's touch input and outputs data of the coordinates (touch coordinates) of the position indicated by this touch input to the controller 180 .

The display unit 112 performs video output. The display unit 112 displays objects such as characters (including symbols), images, and graphics on the screen. For the display unit 112, for example, a liquid crystal display or an organic EL (Electro Luminescence) display is used.

In the touch screen display 110 according to one embodiment, the display unit 112 is provided so as to overlap the touch panel 111 , and the display area of the display unit 112 overlaps the touch panel 111 . However, instead of providing the display unit 112 and the touch panel 111 so as to overlap each other, the display unit 112 and the touch panel 111 may be arranged side by side or separated from each other.

Microphone 120 accepts voice input to information display device 100 . The microphone 120 collects surrounding sounds.

The speaker 130 outputs audio. The speaker 130 outputs voices of telephone calls, information of various programs, and the like.

The camera 140 electronically takes an image using an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). Camera 140 is an in-camera that captures an object facing touch screen display 110 . Information display device 100 may further include an out-camera that captures an object facing the opposite side of touch screen display 110 .

FIG. 2 is a block diagram showing the functional configuration of the information display device 100 according to one embodiment.

As shown in FIG. 2, the information display device 100 includes a touch panel 111, a display unit 112, an audio input unit 121, an audio output unit 131, a camera 140, a sensor 150, a storage unit 160, and a communication interface 170. and a controller 180 .

The touch panel 111 inputs to the controller 180 a signal corresponding to the touch operation with the detected pointer. Further, the touch panel 111 accepts handwriting input with an electronic pen while a handwriting input application, which will be described later, is being executed.

The display unit 112 displays objects such as characters, images, and graphics on the screen based on signals input from the controller 180 . In addition, display unit 112 displays a handwritten trajectory (for example, characters and figures) of which input is received by touch panel 111 during execution of the handwriting input application.

The voice input unit 121 inputs a signal corresponding to the received voice to the controller 180 . Audio input unit 121 includes microphone 120 described above. Also, the voice input unit 121 may be an input interface to which an external microphone can be connected. An external microphone is connected wirelessly or by wire. A microphone connected to the input interface is, for example, a microphone provided in an earphone or the like connectable to the information display device 100 . In one embodiment, an external microphone is provided on the electronic pen.

Audio output unit 131 outputs audio based on a signal input from controller 180 . Audio output unit 131 includes speaker 130 described above. Also, the audio output unit 131 may be an output interface to which an external speaker can be connected. External speakers are connected wirelessly or by wire. A speaker connected to the output interface is, for example, a speaker provided in an earphone or the like connectable to the information display device.

Camera 140 converts the captured image into an electronic signal and inputs it to controller 180 .

The sensor 150 detects acceleration or vibration applied to the information display device 100 and outputs a detection signal corresponding to the detection result to the controller 180 . Sensor 150 includes an acceleration sensor. The acceleration sensor detects the direction and magnitude of acceleration applied to the information display device 100 .

The storage unit 160 stores programs and data. The storage unit 160 is also used as a work area for temporarily storing the processing results of the controller 180 . The storage unit 160 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium.

Also, the storage unit 160 may include multiple types of storage media. The storage unit 160 may include a combination of a portable storage medium such as a memory card, an optical disk, or a magneto-optical disk, and a reading device for the storage medium. The storage unit 160 may include a storage device such as RAM (Random Access Memory) that is used as a temporary storage area.

The programs stored in the storage unit 160 include applications that are executed in the foreground or background, and control programs that support the operation of the applications.

The programs stored in storage unit 160 include a handwriting input application. The handwriting input application is an application that allows the touch panel 111 to receive handwriting input with an electronic pen and causes the display unit 112 to display the locus of handwriting (for example, characters and graphics).

In one embodiment, the handwriting input application has a function of performing voice recognition processing on voice data acquired by the voice input unit 121 . The voice recognition process is a process of recognizing a voice command included in voice data. Each voice command and its corresponding operation content are registered in the storage unit 160 in advance.

Communication interface 170 communicates wirelessly. Wireless communication standards supported by the communication interface 170 include, for example, cellular communication standards such as 2G, 3G, and 4G, short-range wireless communication standards, and the like. Examples of short-range wireless communication standards include IEEE802.11, Bluetooth (registered trademark), IrDA (Infrared Data Association), NFC (Near Field Communication), and WPAN (Wireless Personal Area Network). WPAN communication standards include, for example, ZigBee (registered trademark).

Controller 180 is an arithmetic processing unit. Arithmetic processing units include, but are not limited to, for example, CPU (Central Processing Unit), SoC (System-on-Chip), MCU (Micro Control Unit), FPGA (Field-Programmable Gate Array), and coprocessors. . The controller 180 also includes a GPU (Graphics Processing Unit), a VRAM (Video RAM), etc., and draws various images on the display unit 112 . The controller 180 comprehensively controls the operation of the information display device 100 to implement various functions.

The controller 180 detects whether its own device is connected to an external device. The connection may be wired or wireless. A communication standard for wireless connection is, for example, Bluetooth (registered trademark). Controller 180 communicates with external devices via communication interface 170 . The external device to be connected is, for example, the above earphone, headset, in-vehicle speaker with a microphone, or electronic pen. However, the communication standard for wireless connection and the external device are not limited to these. An example in which the connected external device is an electronic pen will be described below.

The controller 180 executes various controls based on a signal input according to a touch operation or the like detected by the touch panel 111 . For example, the controller 180 uses the audio output unit 131, the display unit 112, and the like to output according to the input signal. Further, the controller 180 executes functions of the information display device 100 and changes settings.

When the user performs handwriting input (touch input) using touch panel 111 while the handwriting input application is running, controller 180 acquires touch coordinate data corresponding to the touch position from touch panel 111 .

A user operates the touch panel 111 with an electronic pen or a finger. Inputs using the touch panel 111 include tap (short press), slide (drag), flick, long touch (long press), and the like. These are sometimes called "touch inputs" or simply "inputs".

A change from a non-touched state to a touched state on the touch panel 111 is called touch-on (pen-down), and a change from a touched state to a non-touched state is called touch-off (pen-up). For input by sliding or flicking, which are continuous touch inputs, the touch panel 111 may output touch coordinate data corresponding to the current touch position in short cycles.

The touch panel 111 outputs touch coordinate data corresponding to a series of touch positions from touch-on (pen-down) to touch-off (pen-up) to the controller 180 . The controller 180 causes the display unit 112 to display the handwritten trajectory indicated by the series of touch coordinate data.

(Configuration of electronic pen)
FIG. 3 is an external view of the electronic pen 200 according to one embodiment.

As shown in FIG. 3 , the electronic pen 200 has a housing 201 , a clip portion 202 , a core 203 and an operation portion 210 .

The housing 201 is configured in a tubular shape. The clip portion 202 is provided on the upper end side of the electronic pen 200 (housing 201). The core body 203 and the operation unit 210 are provided on the lower end side of the electronic pen 200 (housing 201). The operation unit 210 is a button pressed by a finger.

FIG. 4 is a block diagram showing the functional configuration of the electronic pen 200 according to one embodiment.

As shown in FIG. 4 , the electronic pen 200 has an operation section 210 , a microphone 220 , a speaker 230 , a sensor 240 , a storage section 250 , a communication interface 260 and a controller 270 .

The operation unit 210 inputs a signal corresponding to the detected pressing operation to the controller 270 .

The microphone 220 collects surrounding sounds. The microphone 220 receives voice input to the electronic pen 200 and inputs a signal corresponding to the received voice to the controller 270 .

The speaker 230 outputs audio. The speaker 230 outputs telephone voices, information of various programs, etc. by voice. Speaker 230 outputs sound based on the signal input from controller 270 .

The sensor 240 detects acceleration or writing pressure applied to the electronic pen 200 and outputs a detection signal corresponding to the detection result to the controller 270 . Sensor 240 includes an acceleration sensor. The acceleration sensor detects the direction and magnitude of acceleration applied to the electronic pen 200 . Sensor 240 may further include a gyro sensor that detects the angle and angular velocity of information display device 100 .

Sensor 240 further includes a pen pressure sensor. The pen pressure sensor detects the pressure applied to the core 203 (that is, the pen tip) and outputs a signal corresponding to the detection result to the controller 270 .

The storage unit 250 stores programs and data. The storage unit 250 is also used as a work area for temporarily storing the processing results of the controller 270 . The storage unit 250 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium.

Also, the storage unit 250 may include multiple types of storage media. The storage unit 250 may include a combination of a portable storage medium such as a memory card and a storage medium reader. The storage unit 250 may include a storage device such as RAM that is used as a temporary storage area.

Communication interface 260 communicates wirelessly. Wireless communication standards supported by the communication interface 260 include, for example, the above-mentioned cellular communication standards and short-range wireless communication standards.

Controller 270 is an arithmetic processing unit. Examples of processing units include, but are not limited to, CPUs, SoCs, MCUs, FPGAs, and co-processors. The controller 270 comprehensively controls the operation of the electronic pen 200 to implement various functions.

Controller 270 detects whether its own device is connected to information display device 100 . The connection may be wired or wireless. A communication standard for wireless connection is, for example, Bluetooth (registered trademark). Controller 270 communicates with information display device 100 via communication interface 260 .

The controller 270 executes various controls based on a signal input in response to a pressing operation or the like detected by the operation unit 210 . Further, the controller 270 transmits data according to the detection result of the sensor 240 to the information display device 100 when the controller 270 is connected to the information display device 100 .

In one embodiment, controller 270 is capable of performing voice recognition processing. The voice recognition process is a process of recognizing a voice command included in voice data. Each voice command and its corresponding operation content may be registered in the storage unit 250 in advance.

(Operation of handwriting input)
FIG. 5 is a diagram showing an example of a handwriting input screen. Controller 180 of information display device 100 causes touch screen display 110 (display unit 112) to display a handwriting input screen as shown in FIG. 5 during execution of the handwriting input application.

As shown in FIG. 5, the handwriting input screen has a handwriting input area R1, a tool palette area P1, and a color palette area P2.

The handwriting input area R1 is an area for displaying a locus of handwriting. FIG. 5 shows an example of handwriting input by the user using the electronic pen 200 . Generally, a user inputs characters into the handwriting input area R1 with the electronic pen 200 in order to take notes, for example. Characters include numbers and symbols. Alternatively, the user inputs a figure into the handwriting input area R1 with the electronic pen 200 in order to draw an illustration, for example. Figures include curved lines, straight lines, circles, polygons, and the like.

The tool palette area P1 includes buttons B11 to B13 for changing the thickness (line width) of the handwritten trajectory, a button B14 for canceling (undoing) the previous handwritten input, and specifying a portion to be deleted. This is an area for displaying an eraser button B15 for performing a specified operation with the body. On the other hand, the color palette area P2 is an area for displaying color buttons B21 to B25 corresponding to colors such as black, red, blue, green, and yellow.

The user can use the buttons in the tool palette area P1 and the color palette area P2 to select the line color and line width of the handwritten locus (characters, figures, etc.), and to select the surface that constitutes the drawn figure or part thereof. (closed curve) is filled with the desired color. Therefore, every time the display mode (attribute) of the handwritten trajectory is changed, it is necessary to select and change the display mode using the tool palette area P1, the color palette area P2, and the like.

In one embodiment, such display mode changes are enabled by voice. Specifically, in the information display device 100 according to one embodiment, the voice input unit 121 receives voice input including voice commands.

The voice input unit 121 may receive voice input by acquiring voice data including voice commands according to voice input to the microphone 220 of the electronic pen 200 from the electronic pen 200 . By using the microphone 220 of the electronic pen 200, sound can be collected at a position closer to the user. Also, even if the information display device 100 does not have an appropriate microphone, voice input is possible using the microphone 220 of the electronic pen 200 .

When the microphone 220 of the electronic pen 200 is used, the controller 180 of the information display device 100 may perform voice recognition processing for recognizing voice commands, or the controller 270 of the electronic pen 200 may perform voice recognition processing.

Alternatively, the voice input unit 121 may receive voice input by acquiring voice data including a voice command corresponding to voice input to the microphone 120 of the information display device 100 . That is, voice input is performed without using the microphone 220 of the electronic pen 200 . In this case, the controller 180 of the information display device 100 performs voice recognition processing for recognizing voice commands.

The voice command may define a command corresponding to each button in the tool palette area P1 and the color palette area P2.

For example, the voice command corresponding to the button B11 may be "the thinnest line", and the voice command corresponding to the button B13 may be "the thickest line". Alternatively, the line width corresponding to button B11 can be changed to the line width corresponding to button B12 by a relative voice command "thicker line", or the line width corresponding to button B12 can be changed to the line width corresponding to button B13. width, or the line width corresponding to the button B13 may be changed to a thicker line width. That is, the controller 180 of the information display device 100 changes the selected display mode to a display mode similar to the selected display mode and different from the selected display mode based on the relative voice command. may be changed to the display mode of

A voice command corresponding to the button B14 may be "undo". A voice command corresponding to the button B15 may be "eraser".

Buttons B21 to B25 can be voice commands for designating corresponding colors. For example, voice commands such as "Turn the pen blue" and "Turn the pen red" may be used. Alternatively, a gradation from the selected color to red may be specified by a voice command such as "pen gradually red." Alternatively, a gradation from blue to red may be designated by a voice command including designation of a plurality of colors, such as "pen gradually from blue to red." That is, the controller 180 of the information display device 100 specifies one or more similar display modes, and other predetermined voice commands (in the above example, "gradually" corresponds) based on the voice command, The display mode may be changed step by step from the selected display mode or the specified display mode to another display mode of the same type.

The controller 180 controls the display mode of the handwritten trajectory on the display unit 112 according to the voice command received by the voice input unit 121 in the handwriting input state. Here, the display mode includes character color, character size, character thickness, character font, character decoration, figure color, figure size, figure line width, and figure shape. At least one.

Accordingly, when changing the display mode of the handwritten trajectory, the user only has to utter a voice and does not need to perform an input operation for the change using the indicator, so that operability can be improved.

In addition, since the display mode can be changed by voice, the tool palette area P1 and the color palette area P2 shown in FIG. can.

Furthermore, it is also possible to perform advanced operations not included in the tool palette area P1 and the color palette area P2 shown in FIG. 5 by voice. Examples of such advanced manipulations are shown in FIGS.

FIG. 6 is a diagram showing an example of changing the display mode of the handwritten trajectory step by step. As shown in FIG. 6, in a handwriting input state (specifically, a touch state), when the voice input unit 121 receives an input of a voice command C1 "a line from yellow to green gradation", the controller 180 outputs a voice command C1. The command C1 is recognized, and the line color of the handwritten locus is changed in stages from yellow to green.

FIG. 7 is a diagram showing an example of changing the display mode of the handwritten trajectory. As shown in FIG. 7, a handwritten trajectory L1 is drawn. In the handwritten input state, when the voice input unit 121 receives the input of the voice command C2 "straight line", the controller 180 recognizes the voice command C2 and changes the handwritten locus L1 to the straight line L2.

FIG. 8 is a diagram showing another example of changing the display mode of the handwritten trajectory. As shown in FIG. 8, a handwritten trajectory L3 is drawn. In the handwriting input state, when the voice input unit 121 receives the input of the voice command C3 "right angle", the controller 180 recognizes the voice command C3 and changes the handwritten locus L3 to the right angle line L4.

(handwriting input state)
FIG. 9 is a diagram showing a handwriting input state according to one embodiment.

As shown in FIG. 9, the handwriting input state includes a touch state in which electronic pen 200 is touching the display surface of touch screen display 110 (display unit 112).

The controller 180 controls the display mode of the handwritten trace on the display unit 112 according to the voice command received by the voice input unit 121 in the touch state. As a result, it is possible to change the display mode of the trajectory of the handwriting by the voice produced during the handwriting input.

For example, when the voice input unit 121 receives an input of a voice command “turn the pen red” in the touch state, the controller 180 recognizes this voice command and changes the color to red from the middle of the handwritten locus.

Also, when the voice input unit 121 receives an input of a voice command “to the thickest line” in the touch state, the controller 180 recognizes this voice command and makes the line width the thickest from the middle of the handwritten locus. Change to line.

Alternatively, when the voice input unit 121 receives an input of a voice command “thicker” in the touch state, the controller 180 recognizes this voice command and increases the line width by one step from the middle of the handwritten locus. change. After that, when the voice input unit 121 receives the input of the voice command "Thicker" again before the pen is raised, the controller 180 recognizes this voice command and thickens the line width by one step from the middle of the handwritten locus. Change to line. In this way, when the voice command input is repeatedly received, the controller 180 changes the handwriting trajectory step by step.

After that, when the voice input unit 121 receives the input of the voice command "thinner" again before the pen is raised, the controller 180 recognizes this voice command, and reduces the line width by one step from the middle of the handwritten trajectory. change to

As a result, the line width of the handwritten trajectory can be changed stepwise during handwriting input. Here, when changing in steps, the line width may be changed gradually instead of being changed abruptly.

Alternatively, when the line color is blue in the touch state, when the voice input unit 121 receives the input of the voice command "to be more red", the controller 180 recognizes this voice command and , make the line color one step closer to red (eg, purple with stronger blue).

After that, when the voice input unit 121 receives the input of the voice command "more red" again before the pen is raised, the controller 180 recognizes this voice command and changes the line color by one step from the middle of the handwritten locus. Closer to red (e.g. plain purple).

After that, when the voice input unit 121 receives the input of the voice command "more red" again before the pen is raised, the controller 180 recognizes this voice command and changes the line color to red by one level from the middle of the handwritten locus. (for example, purple with strong red).

As a result, the line color of the handwritten trajectory can be changed in stages during handwriting input. Here, when changing the line color in stages, the line color may be gradually changed instead of being changed abruptly, and drawn with a gradation from blue to red.

Alternatively, when the voice input unit 121 accepts the input of the voice command "this is important" in the touch state during character input, the controller 180 recognizes this voice command and reads the handwritten text for a predetermined time or length before and after it. You may change the color of the trajectory to red or add a highlight (marker). Such a voice command can be said to be a voice command for uniformly changing (specifically, highlighting) the display mode of a predetermined range before and after handwriting input.

Alternatively, when the voice input unit 121 receives the input of the voice command "This is important" in the touch state during character input, the controller 180 recognizes this voice command, and recognizes the voice command from the pen-down timing t2. The color of the handwritten trajectory to the timing may be changed to red or highlighted. Such a voice command can be said to be a voice command for uniformly changing (more specifically, highlighting) the display mode of the previous predetermined range during handwriting input.

Alternatively, when the voice input unit 121 receives the input of the voice command "Important from here" in the touch state during character input, the controller 180 recognizes this voice command, and starts the pen-up operation from the timing when the voice command is recognized. The color of the handwritten trajectory up to timing t3 may be changed to red or highlighted. Such a voice command can be said to be a voice command for uniformly changing (specifically, highlighting) the display mode of a predetermined range immediately after handwritten input.

However, after inputting the voice command "Important from here", if the voice command "Important so far" is input before the pen is raised, the voice command "Important from here" is recognized. The handwritten trajectory up to the timing of recognition of the voice command may be changed to red or highlighted.

A handwriting input state may include a non-touch state immediately following a touch state. Specifically, the handwriting input state is defined as a state of non-handwriting during a period from timing t3 when electronic pen 200 moves away from the display surface of touch screen display 110 (display unit 112) (that is, pen-up timing) until a predetermined time elapses. It may also include a touch state. As a result, it is possible to change the display mode of the handwritten trajectory by the voice produced immediately after pen-up.

For example, when the voice input unit 121 receives a voice command input of “undo” immediately after pen-up, the controller 180 recognizes this voice command, and the character or figure input during the touch state is changed. A handwritten trace may be deleted.

Alternatively, when the voice input unit 121 receives the input of the voice command “eraser” immediately after the pen is up, the controller 180 recognizes this voice command and writes the handwritten characters or figures input during the touch state. A mode for deleting the trajectory may be set, a mark (or an eraser icon) for the electronic pen 200 to designate the deletion location may be displayed, and the location designated by the electronic pen 200 may be deleted.

Alternatively, when the voice input unit 121 receives an input of a voice command “turn the pen red” immediately after the pen is raised, the controller 180 recognizes this voice command and recognizes the characters, figures, etc., input during the touch state. , change the color of the handwritten trace to red.

Further, when the voice input unit 121 receives the input of the voice command "to the thickest line" immediately after the pen-up, the controller 180 recognizes this voice command, and Change the line width of handwritten traces such as to the thickest line.

A handwriting input state may include a non-touch state immediately preceding a touch state. Specifically, the handwriting input state may include a non-touch state from timing t2 at which electronic pen 200 touches the display surface (that is, pen-down timing) to a predetermined time before. As a result, it is possible to change the display mode of the handwritten trajectory by the voice uttered immediately before the pen-up.

For example, if the voice input unit 121 accepts the input of a voice command “turn the pen red” just before pen down, the controller 180 recognizes this voice command and recognizes characters, graphics, etc. input during the touch state. Change the color of the handwritten locus to red.

In addition, when the voice input unit 121 receives the input of the voice command "to the thickest line" just before the pen-up, the controller 180 recognizes this voice command and recognizes the character or figure input during the touch state. Make the line width of the handwritten trajectory such as the thickest line.

(Example of operation flow)
FIG. 10 is a diagram showing an example of the operation flow of the information display device 100 and the electronic pen 200. As shown in FIG. A wireless connection (for example, a short-range wireless communication connection) is set between the information display device 100 and the electronic pen 200 .

As shown in FIG. 10, in step S101, the controller 180 of the information display device 100 starts voice recognition processing. By starting voice recognition processing, it becomes possible to recognize a predetermined voice command in the handwriting input state.

The controller 180 of the information display device 100 may start speech recognition processing in response to activation of the handwriting input application. As a result, speech recognition processing can be started at an appropriate timing.

Alternatively, the controller 180 of the information display device 100 performs voice recognition processing in response to a handwriting input application being activated and a pressing operation (first operation) being performed on the operation unit 210 of the electronic pen 200. may be started. The user presses the operation unit 210 of the electronic pen 200 before uttering a voice corresponding to the voice command. As a result, speech recognition processing can be started at a more appropriate timing.

The controller 180 of the information display device 100 may start voice recognition processing when the handwriting input application is activated and the sensor 240 (pen pressure sensor) of the electronic pen 200 detects pen pressure. . Alternatively, the controller 180 of the information display device 100 may start voice recognition processing when the handwriting input application is activated and the touch panel 111 detects a touch input (pen down). Accordingly, even if the user does not press the operation unit 210 of the electronic pen 200, the voice recognition process can be started at an appropriate timing.

In step S<b>102 , voice is input to the microphone 220 of the electronic pen 200 . The microphone 220 of the electronic pen 200 converts this voice into voice data and outputs it to the controller 270 of the electronic pen 200 .

In step S<b>103 , the controller 270 of the electronic pen 200 transmits voice data input from the microphone 220 of the electronic pen 200 to the information display device 100 via the communication interface 260 of the electronic pen 200 . The voice input unit 121 of the information display device 100 receives voice input by acquiring voice data from the electronic pen 200 and outputs this voice data to the controller 180 of the information display device 100 .

In step S104, the controller 180 of the information display device 100 performs voice recognition processing on the voice data input from the voice input unit 121, and confirms whether or not the voice data includes a voice command.

If a voice command is included, in step S105, the controller 180 of the information display device 100 controls the display mode of the handwritten trajectory on the display unit 112 according to the voice command.

Note that the controller 180 of the information display device 100 terminates the voice recognition processing when the handwriting input application is terminated.

Alternatively, the controller 180 of the information display device 100 may enable voice recognition while the operation unit 210 of the electronic pen 200 is being pressed. In this case, the controller 180 may end the voice recognition process in response to the pressing release (second operation) on the operation unit 210 .

Alternatively, the controller 180 of the information display device 100 may enable voice recognition in the touch state. In this case, the controller 180 of the information display device 100 may terminate the voice recognition process in response to the pen pressure sensor of the electronic pen 200 no longer detecting pressure. Alternatively, the controller 180 of the information display device 100 may terminate the voice recognition process when the touch panel 111 detects the pen-up.

FIG. 11 is a diagram showing another example of the operation flow of the information display device 100 and the electronic pen 200. As shown in FIG. A wireless connection (for example, a short-range wireless communication connection) is set between the information display device 100 and the electronic pen 200 . Here, differences from the operation shown in FIG. 10 will be mainly described.

As shown in FIG. 11, in step S201, the controller 270 of the electronic pen 200 starts speech recognition processing.

The controller 270 of the electronic pen 200 may start voice recognition processing when notified by the information display device 100 that the handwriting input application has been activated. Alternatively, the controller 270 of the electronic pen 200 executes the voice recognition process in response to the handwriting input application being activated and the operation unit 210 of the electronic pen 200 being pressed (first operation). may start. The controller 270 of the electronic pen 200 may start voice recognition processing when the handwriting input application is activated and the sensor 240 (pen pressure sensor) of the electronic pen 200 detects pen pressure.

In step S<b>202 , voice is input to the microphone 220 of the electronic pen 200 . The microphone 220 of the electronic pen 200 converts this voice into voice data and outputs it to the controller 270 of the electronic pen 200 .

In step S203, the controller 270 of the electronic pen 200 performs voice recognition processing on the voice data input from the voice input unit 121, and confirms whether the voice data includes a voice command.

If a voice command is included, the controller 270 of the electronic pen 200 transmits data corresponding to the voice command to the information display device 100 via the communication interface 260 of the electronic pen 200 in step S204. The voice input unit 121 of the information display device 100 receives voice input by acquiring data corresponding to the voice command from the electronic pen 200 and outputs data corresponding to the voice command to the controller 180 of the information display device 100 .

In step S<b>205 , the controller 180 of the information display device 100 controls the display mode of the handwritten trajectory on the display section 112 according to the data corresponding to the voice command input from the voice input section 121 .

Note that the controller 270 of the electronic pen 200 may terminate the voice recognition process when notified by the information display device 100 that the handwriting input application has terminated.

Alternatively, the controller 270 of the electronic pen 200 may enable voice recognition while the operation unit 210 of the electronic pen 200 is being pressed. In this case, the controller 270 of the electronic pen 200 may end the voice recognition process in response to the pressing release (second operation) of the operation unit 210 of the electronic pen 200 .

Alternatively, the controller 270 of the electronic pen 200 may enable voice recognition in the touch state. In this case, the controller 270 of the electronic pen 200 may terminate the voice recognition process in response to the writing pressure sensor of the electronic pen 200 no longer detecting pressure.

(Other embodiments)
In the above embodiment, the case where the electronic pen 200 can be stored in the information display device 100 was not particularly mentioned. It may be possible to store in

A program that causes a computer to execute each process performed by the information display device 100 may be provided. The program may be recorded on a computer readable medium. A computer readable medium allows the installation of the program on the computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as CD-ROM or DVD-ROM.

An embodiment has been described in detail above with reference to the drawings, but the specific configuration is not limited to the one described above, and various design changes can be made without departing from the spirit of the invention. .

100: Information display device 101: Housing 110: Touch screen display 111: Touch panel 112: Display unit 120: Microphone 121: Audio input unit 130: Speaker 131: Audio output unit 140: Camera 150: Sensor 160: Storage unit 170: Communication Interface 180 : Controller 200 : Electronic pen 201 : Housing 202 : Clip section 203 : Core body 210 : Operation section 220 : Microphone 230 : Speaker 240 : Sensor 250 : Storage section 260 : Communication interface 270 : Controller

Claims (13)

a touch panel that accepts handwriting input with an electronic pen having a pen pressure sensor;
a communication interface that receives the pressure detected by the pen pressure sensor from the electronic pen;
a display unit that displays a handwritten trajectory received by the touch panel;
a voice input unit that receives voice input including voice commands;
After the handwriting input application is activated, in response to the pen pressure sensor detecting pressure , speech recognition processing is started, and in accordance with the voice command received by the speech recognition processing, the handwritten trajectory is displayed on the display unit. and a controller that controls the information display device. The electronic pen has a microphone,
The information display device according to claim 1, wherein the voice input unit accepts the voice input by acquiring voice data including a voice command corresponding to voice input to the microphone from the electronic pen. 2. The controller according to claim 1, wherein the display mode is controlled according to a voice command received by the voice input unit during a predetermined period of time from when the electronic pen is separated from the touch panel. information display device. The information display device according to claim 1, wherein the controller controls the display mode according to a voice command received by the voice input unit immediately after the electronic pen is separated from the touch panel. The display unit displays characters as the handwritten trajectory,
5. Any one of claims 1 to 4, wherein the controller changes the mode of characters displayed by the display unit according to a voice command received by the voice input unit after the electronic pen leaves the touch panel. The information display device according to the item. The display unit displays a figure as the trajectory of the handwriting,
5. Any one of claims 1 to 4, wherein the controller changes the form of the figure displayed by the display unit according to a voice command received by the voice input unit after the electronic pen leaves the touch panel. The information display device according to the item. 5. The controller according to any one of claims 1 to 4, wherein the controller deletes the trace of handwriting on the display unit in response to a voice command received by the voice input unit after the electronic pen leaves the touch panel. Information display device as described. After the electronic pen leaves the touch panel, the controller
recognizing a voice command included in the voice input received by the voice input unit;
The information display device according to any one of claims 1 to 7, wherein the display mode of the handwritten trajectory on the display unit is changed in response to recognition of a predetermined voice command. The information display device according to claim 8, wherein the controller terminates the speech recognition processing in response to termination of the handwriting input application. 9. The information display device according to claim 8, wherein the controller terminates the voice recognition processing when the writing pressure sensor no longer detects pressure. An electronic pen functioning as the indicator according to any one of claims 1 to 10. A touch panel that receives handwritten input by an electronic pen having a pen pressure sensor, a communication interface that receives pressure detected by the pen pressure sensor from the electronic pen, and a display unit that displays the locus of handwriting received by the touch panel. In the information display device having
After the handwriting input application is activated, voice recognition processing is started according to the pressure detected by the writing pressure sensor, and the display mode of the handwritten trajectory on the display unit is controlled according to the voice command received by the voice recognition processing. A display control program that causes a control step to be executed. A display control method for use in an information display device having a touch panel that receives handwriting input with an electronic pen that has a pen pressure sensor and is communicable with the information display device,
a voice input step of accepting voice input including voice commands in the handwriting input state;
performing a voice recognition process to recognize the voice commands contained in the voice input;
a display step of displaying a handwritten trajectory received by the touch panel;
a control step of controlling a display mode of the handwritten trajectory displayed in the display step according to the voice command recognized in the step of performing the voice recognition process ;
The display control method, wherein the step of performing the speech recognition processing includes the step of starting the speech recognition processing in response to a handwriting input application being activated and the writing pressure sensor detecting pressure.

Images