WO2023126664A1 - System and method for providing a web browser online user interface with haptic feedback for an automotive setting - Google Patents

System and method for providing a web browser online user interface with haptic feedback for an automotive setting Download PDF

Info

Publication number
WO2023126664A1
WO2023126664A1 PCT/IB2021/062477 IB2021062477W WO2023126664A1 WO 2023126664 A1 WO2023126664 A1 WO 2023126664A1 IB 2021062477 W IB2021062477 W IB 2021062477W WO 2023126664 A1 WO2023126664 A1 WO 2023126664A1
Authority
WO
WIPO (PCT)
Prior art keywords
haptic
display
image
web browser
pattern
Prior art date
Application number
PCT/IB2021/062477
Other languages
French (fr)
Inventor
Pedro Miguel DE SOUSA BERNARDO
Vera Patricia CASTRO VILAS BOAS
Nélson Bruno MARTINS MARQUES COSTA
Manuel Sebastião MARQUES COUTINHO
Adriano Dídimo MACHADO CARVALHO
Rosane SCHONBLUM SAMPAIO
Marina MACHADO
Lisandra Maria ARAÚJO TEIXEIRA
Emanuel TEIXEIRA SILVA
José Pedro CORREIA DIAS
Original Assignee
Bosch Car Multimedia Portugal S.A
Universidade Do Minho
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bosch Car Multimedia Portugal S.A, Universidade Do Minho filed Critical Bosch Car Multimedia Portugal S.A
Priority to EP21854695.0A priority Critical patent/EP4457586A1/en
Publication of WO2023126664A1 publication Critical patent/WO2023126664A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03547—Touch pads, in which fingers can move on a surface
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B6/00—Tactile signalling systems, e.g. personal calling systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41422—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance located in transportation means, e.g. personal vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/013—Force feedback applied to a game
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/014—Force feedback applied to GUI
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4782—Web browsing, e.g. WebTV

Definitions

  • the present disclosure belongs to the technical field of automotive passenger interfaces.
  • the present disclosure relates to a system and method for providing a web browser online interface comprising a display surface with haptic feedback for a user in an autonomous vehicle setting.
  • Document US2017153702 providing and/or determining haptic feedback may include, responsive to determining that a user is proximate to a haptic feedback device, determining, using a processor, a location of the haptic feedback device, and receiving, using the processor, a plurality of external factors determined from the location of the haptic feedback device, biometric data for the user, and social networking data for the user.
  • the method may also include determining, using the processor, haptic feedback for use by the haptic feedback device by applying the external factors to a haptic feedback template of an object.
  • Document US2018046250 describes frame structure with downlink/uplink subframe configuration and channel hopping scheme for unlicensed narrowband Internet-of-Things (loT) systems.
  • An apparatus operable for unlicensed narrowband transmission to support loT service is disclosed.
  • the apparatus includes baseband circuitry to select a transmission channel within an unlicensed narrow band for downlink transmission of a discovery reference signal (DRS), and for channel hopping, to select, according to the DRS, a communication channel within the unlicensed narrow band for downlink data and uplink data.
  • the DRS includes a primary synchronization signal (PSS), a secondary synchronization signal (SSS) and physical broadcast channel (PBCH) content.
  • PSS primary synchronization signal
  • SSS secondary synchronization signal
  • PBCH physical broadcast channel
  • the baseband circuitry is further to demodulate a received signal received via radio frequency (RF) circuitry through the communication channel over an uplink frame, and to modulate a transmitting signal to be transmitted via the RF circuitry through
  • Document US20200264772 discloses a method for providing haptic feedback to an operator of a touch-sensitive display device.
  • the method includes providing an operator interface to be represented in the display device as image data; the operator interface having at least one button assigned to a function to be controlled, graphically displayed in the operator interface, and graphically delimited from the rest of the operator interface; analysing the image data for a presence and a position of shapes for representing a button; determining an intended display region of a shape, which is to be represented and is identified as a button; representing the image data of the operator interface in the touch-sensitive display device; outputting haptic feedback in response to detection of contact with the touch-sensitive display device in the region of a surface of the touch-sensitive display device which is assigned to a display region of a shape identified as a button.
  • the present disclosure discloses on making use of a smart surface, composed of an insulative layer, fitted atop a transparent and conductive electrode sheet layer, which, when charged with an electric current and when in contact with the human skin, produces a frictional stimulus by controlling the electric current.
  • This layer is in turn fitted atop a capacitive-based touch sensitive layer, fitted atop a layer of LED lights, which in turn are fitted atop a layer of vibrotactile actuators.
  • the layer of LEDs is optional and when not present the touch-sensitive layer is fitted atop a layer of vibrotactile actuators.
  • the present disclosure can be applied to a train carriage, airplane fuselage or other passenger compartment.
  • the present disclosure is employable in level 5 autonomous vehicles.
  • This smart surface can be employed either as: a) the interactive surface interface of a display device to which it is physically connect, e.g., the smart surface is part of the device's display; or b) as the interactive surface interface of a display device to which it is not physically connected, e.g., touch surface interface communicating wirelessly with a display device in close proximity.
  • the capacitive-based touch sensitive layer can be made of a variety of materials, depending on the intended use case and aesthetic appeal. [0019] In an embodiment, the capacitive-based touch sensitive layer is made of transparent glass - display device; translucent plastic - interactive surface interface for an external device, using the LED lights layer as the display component of said device.
  • the smart surface can generate both frictional and vibrotactile haptic sensations.
  • Frictional haptic sensations are generated thanks to the insulative and conductive layers, which allow for friction between the smart surface and the user's skin to be modulated through electrostatic actuation, while the skin is in motion.
  • Vibrotactile haptic sensations are generated thanks to the vibrotactile actuators, which can generate vibrations strong enough to be felt through the user's skin, either stationary or in motion, when in direct contact with the smart surface.
  • the smart surface is used as the interactive interface for an external device, haptic feedback sensations, caused by the external device's software, can be generated on it, as long as the haptic actuation technology that is present on the smart surface is both capable of interpreting the software's information and of generating such sensations.
  • this smart surface is to be fitted on at least two regions inside the vehicle: as the display surface of the central stack display (these can provide vibrotactile feedback, while this application would make use of both frictional and vibrotactile feedback), where it can serve as said device's interactive display surface interface (kind of like how a laptop's mousepad controls the mouse shown on the screen, interacting with the interactive surface of the armrest would control what is shown on the center stack display device); and on a portion of the surface of the seat's armrest, where it can act as an external interactive (Both communicate through the vehicle's internal infotainment software system, which is the main controller of both ends) surface interface for the center stack display device.
  • the smart surface is connected to the internal electronics of the vehicle's systems, be it through the centre stack display device, which is directly connected and controls the vehicle's infotainment system, or, in the case of the armrest's smart surface, directly connected to the vehicle's infotainment system.
  • the centre stack display device which is directly connected and controls the vehicle's infotainment system, or, in the case of the armrest's smart surface, directly connected to the vehicle's infotainment system.
  • commands and interactions inputted through the armrest's smart surface can be interpreted, and visualized on the centre stack display.
  • information displayed on the centre stack display that possesses commands for haptic feedback sensations can also be sent to and actuated on the armrest's smart surface surface interface for the centre stack display device.
  • the haptic feedback web browser app can be installed and accessed through several different devices, such as the vehicle's infotainment system or the user's personal smartphone. Additionally, in order to access and display web content, the device has access to the Internet and is able to connect to it.
  • haptic feedback sensations make use of frictional and/or vibrotactile stimuli, which can be generated through electrostatic and vibrotactile actuator, respectively.
  • an adequate external interactive surface interface can be used to generate said stimuli.
  • said surface should be somehow able to connect to the device, either through a wired or a wireless connection, and should also possess either one or both types of actuators necessary to generate the intended hapticfeedback stimuli, in orderto receive, interpret, and be able to generate the intended haptic feedback sensations.
  • the visual content of the app is always displayed on the main interface screen.
  • the app can access and present web content to users, such as text and images.
  • said web content is linked with a corresponding haptic feedback sensation
  • said feedback is also generated on the device's smart surface so that users can feel it, provided that the actuators present on said smart surface can generate the intended feedback sensations (e.g., textures).
  • the app is displayed on the center stack display device.
  • the haptic feedback sensations can be felt during user tactile interactions with either the display device's smart surface, or the armrest' smart surface.
  • the haptic feedback information associated with each web content is to be designed and/or provided by the address hosting said web content. This information is then downloaded by the app, which assesses if the connected devices/surfaces can reproduce the haptic feedback sensations. If yes, the haptic feedback sensations are generated either on all connected and available surfaces, or on those surfaces which the user has previously selected. If not, haptic feedback sensations are not generated.
  • a system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
  • system further can comprise said display for displaying images of patterns or objects, wherein the haptic surface is superimposed on the display.
  • the haptic surface is arranged to communicate wirelessly with the electronic data processor.
  • said display is arranged to communicate wirelessly with the electronic data processor.
  • the automotive setting is an autonomous vehicle and the display is a centre stack display of the autonomous vehicle.
  • the automotive setting is an autonomous vehicle and comprising a secondary display which is a centre stack display of the autonomous vehicle, wherein the electronic data processor is further configured to: provide the web browser online user interface on the secondary display for displaying images of patterns or objects; display the image on the web browser interface on said secondary display.
  • the secondary display comprises:
  • a haptic surface comprising actuators for exhibiting tactile textures and superimposed on said secondary display, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; wherein the electronic data processor is further configured to: drive the haptic layers of the secondary display to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed on said secondary display.
  • said two displays are arranged to communicate wirelessly with the electronic data processor.
  • the electronic data processor is further configured to select either a frictional or a vibrotactile haptic feedback as a function of the pattern or object to be displayed.
  • the capacitive-based touch sensitive layer is made of translucent plastic comprising LEDs underneath it.
  • [0048] display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
  • Figure 1 Schematic representation of an embodiment of a home/quick access page layout.
  • Figure 2 Schematic representation of an embodiment of a clothing shop website layout.
  • Figure 3 Schematic representation of an embodiment of a furniture shop website layout.
  • Figure 4 Schematic representation of an embodiment of an interface with a capacitive, haptic feedback on armrest and central display.
  • the present disclosure relates to a system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
  • the disclosure also relates to the operation method thereof.
  • Figure 1 shows a schematic representation of an embodiment of a home/quick access page layout search or URL navigation field, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen;
  • 102 represents an image preview of a bookmarked webpage, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen;
  • 103 represents a button to add a new bookmark to the home/quick access page layout, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen.
  • Figure 2 shows a schematic representation of an embodiment of a clothing store's item selection webpage, where: 101 represents a preview window showing how the clothing item looks like with the selected fabric material, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents the selection button for users to select which material they want to buy; 103 represents a zoomed in area of the selected fabric material is previewed, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen.
  • Figure 3 shows a schematic representation of an embodiment of a furniture store's item selection webpage, where: 101 represents a preview window showing how the furniture item looks like with the selected material, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents the selection button for users to select which material they want to buy; 103 represents a zoomed in area of the selected material is previewed, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen.
  • Figure 4 shows a schematic representation of an embodiment of a capacitive haptic surface on armrest, where: 11 represents a passenger, 12 represents the seat, 13 represents the armrest with the capacitive, haptic surface, and 14 represents the center stack display that the capacitive, haptic surface of the armrest is in contact with.
  • a system can comprise a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
  • FIG. 2 a schematic representation of an embodiment of a clothing shop website layout.
  • FIG. 3 a schematic representation of an embodiment of a furniture shop website layout.
  • examples of textures that participants subjectively report feeling during interaction with electrode devices include: wood, leather, paper, painted wall, rubber, and viscous liquid (Bau, Olivier, and Ivan Poupyrev. "REVEL: tactile feedback technology for augmented reality.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed. Operation method thereof.

Description

D E S C R I P T I O N
SYSTEM AND M ETHOD FOR PROVI DI NG A WEB BROWSER ON LIN E USER I NTERFACE WITH HAPTIC FEEDBACK FOR AN AUTOMOTIVE SETTI NG
TECHNICAL FIELD
[0001] The present disclosure belongs to the technical field of automotive passenger interfaces. In particular, the present disclosure relates to a system and method for providing a web browser online interface comprising a display surface with haptic feedback for a user in an autonomous vehicle setting.
BACKGROUND
[0002] Document US2017153702 providing and/or determining haptic feedback may include, responsive to determining that a user is proximate to a haptic feedback device, determining, using a processor, a location of the haptic feedback device, and receiving, using the processor, a plurality of external factors determined from the location of the haptic feedback device, biometric data for the user, and social networking data for the user. The method may also include determining, using the processor, haptic feedback for use by the haptic feedback device by applying the external factors to a haptic feedback template of an object.
[0003] Document US2018046250 describes frame structure with downlink/uplink subframe configuration and channel hopping scheme for unlicensed narrowband Internet-of-Things (loT) systems. An apparatus operable for unlicensed narrowband transmission to support loT service is disclosed. The apparatus includes baseband circuitry to select a transmission channel within an unlicensed narrow band for downlink transmission of a discovery reference signal (DRS), and for channel hopping, to select, according to the DRS, a communication channel within the unlicensed narrow band for downlink data and uplink data. The DRS includes a primary synchronization signal (PSS), a secondary synchronization signal (SSS) and physical broadcast channel (PBCH) content. The baseband circuitry is further to demodulate a received signal received via radio frequency (RF) circuitry through the communication channel over an uplink frame, and to modulate a transmitting signal to be transmitted via the RF circuitry through the communication channel over a downlink frame.
[0004] Document US20200264772 discloses a method for providing haptic feedback to an operator of a touch-sensitive display device. The method includes providing an operator interface to be represented in the display device as image data; the operator interface having at least one button assigned to a function to be controlled, graphically displayed in the operator interface, and graphically delimited from the rest of the operator interface; analysing the image data for a presence and a position of shapes for representing a button; determining an intended display region of a shape, which is to be represented and is identified as a button; representing the image data of the operator interface in the touch-sensitive display device; outputting haptic feedback in response to detection of contact with the touch-sensitive display device in the region of a surface of the touch-sensitive display device which is assigned to a display region of a shape identified as a button.
[0005] These facts are disclosed in order to illustrate the technical problem addressed by the present disclosure.
GENERAL DESCRIPTION
[0006] The present disclosure discloses on making use of a smart surface, composed of an insulative layer, fitted atop a transparent and conductive electrode sheet layer, which, when charged with an electric current and when in contact with the human skin, produces a frictional stimulus by controlling the electric current. This layer is in turn fitted atop a capacitive-based touch sensitive layer, fitted atop a layer of LED lights, which in turn are fitted atop a layer of vibrotactile actuators. The layer of LEDs is optional and when not present the touch-sensitive layer is fitted atop a layer of vibrotactile actuators.
[0007] Recently, some auto manufactures have started to incorporate web browsing applications (apps) in their vehicle's infotainment systems (e.g., Tesla®, n.d.). The integration and use of this kind of app in a vehicle's infotainment system, while the vehicle is being driven, raises serious concerns regarding the risks of driver distraction, especially if the one using the app is the driver himself/herself.
[0008] However, as the automotive industry evolves, it is expected that users will need to spend less time actively driving their vehicles, which will in turn result in them having more free time for themselves during their commutes.
[0009] Most in-vehicle infotainment displays and surfaces are not equipped with haptic feedback capabilities, so the interactions between users and these devices are very basic.
[0010] Haptic feedback in devices and surfaces has commonly been achieved through vibrotactile stimuli, generated from mechanical actuators (e.g., EP3582074A1).
[0011] More recently, technologies have emerged that make use of other actuation types, such as electro vibration, to generate other types of tactile feedback beyond just vibrations (e.g., US10768749B2). The technology for creating richer display and surface interactions is available, by delivering different haptic feedback sensations, such as different texture sensations or different vibrations, to users while they interacted with the display surface with their fingers. However, there is a lack of apps that exploit the capabilities of these interface to the maximum.
[0012] With the expected incoming increase of autonomous vehicles that will become available to the general consumer market, users of such vehicles will find themselves with more free time during their commutes. Users might choose to engage in a variety of activities to make use of this free time, for leisure or work purposes, including accessing the internet for shopping, browsing social media, or looking up information.
[0013] To the best of our knowledge, not many web browsing apps, developed mainly for use inside or alongside a vehicle's infotainment systems, currently exist. For example, the Tesla Web Browser, developed for use on some Tesla Vehicles, allows users to navigate the web using the vehicle's centre stack display (Tesla®, 2018). However, this centre stack display does not provide haptic feedback to users, neither does the web browser application incorporate information and instructions for devices to interpret and deliver haptic feedback sensations to users. Moreover, this Web Browser application possesses several limitations in order to minimize driver distraction during the task of driving. As display surface technology develops, more focus is being given to display devices and surfaces that can provide new and innovative ways of enriching the interactions users have with them, such as generating richer haptic feedback sensations through the use of frictional and vibrotactile stimuli. In level 5 of autonomous vehicles, i.e., in full automation, concerns regarding driver distraction are no longer present, as the task of driving the vehicle is taken up by the vehicle's system, and the driver becomes just another passenger in the vehicle. Therefore, passengers may choose to use the time during their commutes for other tasks, such as reading, watching media, browsing the internet, or online shopping. Therefore, it can be expected that users will pay more attention to the characteristics and quality of their interactions with different elements of the vehicle, such as different surfaces and the central stack display. For this reason, by adding the capability of generating different haptic feedback sensations on these surfaces and display, and developing software that can make use of said feedback sensations, a more enjoyable and pleasurable experience can be delivered to users of these autonomous vehicles.
[0014] The present disclosure can be applied to a train carriage, airplane fuselage or other passenger compartment.
[0015] It is disclosed a web browsing interface, with the aim of it being used primarily in level 5 autonomous vehicle infotainment displays, that makes use of the richer interaction opportunities that some haptic feedback display and surface technologies currently offer, while also planning for what these technologies and other haptic feedback technologies are able to provide users with.
[0016] The present disclosure is employable in level 5 autonomous vehicles.
[0017] This smart surface can be employed either as: a) the interactive surface interface of a display device to which it is physically connect, e.g., the smart surface is part of the device's display; or b) as the interactive surface interface of a display device to which it is not physically connected, e.g., touch surface interface communicating wirelessly with a display device in close proximity.
[0018] The capacitive-based touch sensitive layer can be made of a variety of materials, depending on the intended use case and aesthetic appeal. [0019] In an embodiment, the capacitive-based touch sensitive layer is made of transparent glass - display device; translucent plastic - interactive surface interface for an external device, using the LED lights layer as the display component of said device.
[0020] In this present disclosure, the smart surface can generate both frictional and vibrotactile haptic sensations.
[0021] Frictional haptic sensations are generated thanks to the insulative and conductive layers, which allow for friction between the smart surface and the user's skin to be modulated through electrostatic actuation, while the skin is in motion.
[0022] Vibrotactile haptic sensations, in turn, are generated thanks to the vibrotactile actuators, which can generate vibrations strong enough to be felt through the user's skin, either stationary or in motion, when in direct contact with the smart surface. When the smart surface is used as the interactive interface for an external device, haptic feedback sensations, caused by the external device's software, can be generated on it, as long as the haptic actuation technology that is present on the smart surface is both capable of interpreting the software's information and of generating such sensations.
[0023] For the present disclosure being discussed in this document, this smart surface is to be fitted on at least two regions inside the vehicle: as the display surface of the central stack display (these can provide vibrotactile feedback, while this application would make use of both frictional and vibrotactile feedback), where it can serve as said device's interactive display surface interface (kind of like how a laptop's mousepad controls the mouse shown on the screen, interacting with the interactive surface of the armrest would control what is shown on the center stack display device); and on a portion of the surface of the seat's armrest, where it can act as an external interactive (Both communicate through the vehicle's internal infotainment software system, which is the main controller of both ends) surface interface for the center stack display device. In both cases, the smart surface is connected to the internal electronics of the vehicle's systems, be it through the centre stack display device, which is directly connected and controls the vehicle's infotainment system, or, in the case of the armrest's smart surface, directly connected to the vehicle's infotainment system. In this way, commands and interactions inputted through the armrest's smart surface, can be interpreted, and visualized on the centre stack display. On the other hand, information displayed on the centre stack display that possesses commands for haptic feedback sensations can also be sent to and actuated on the armrest's smart surface surface interface for the centre stack display device.
[0024] The haptic feedback web browser app can be installed and accessed through several different devices, such as the vehicle's infotainment system or the user's personal smartphone. Additionally, in order to access and display web content, the device has access to the Internet and is able to connect to it.
[0025] The interaction between user and app is more fruitful when done through the previously disclosed smart surface, especially when said interaction occurs inside the vehicle, during the user's commute.
[0026] The herein described haptic feedback sensations make use of frictional and/or vibrotactile stimuli, which can be generated through electrostatic and vibrotactile actuator, respectively.
[0027] When the app is accessed through a device without a surface that can produce one or both of the intended haptic feedback stimulus types, an adequate external interactive surface interface can be used to generate said stimuli. For this, said surface should be somehow able to connect to the device, either through a wired or a wireless connection, and should also possess either one or both types of actuators necessary to generate the intended hapticfeedback stimuli, in orderto receive, interpret, and be able to generate the intended haptic feedback sensations.
[0028] Although one of the main features of this app is the employment of haptic feedback sensations during the interactions between it and the user, there are no impediments regarding installing and using this app on devices without actuators that can produce the stimuli types from which this app benefits, as long as users keep in mind that the full intended experience cannot be achieved in said devices.
[0029] There is also no impediment in the user choosing to use an external interactive surface interface to interact with the app when displayed on a device which also possesses a screen surface with haptic feedback capabilities. For example, users might choose to interact with the app through the armrest' smart surface, while leaning back on their seat, for extra comfort, while others might prefer to interact with it through the centre stack display itself, interacting "directly" with the visual content displayed on the screen.
[0030] In one embodiment, the visual content of the app is always displayed on the main interface screen.
[0031] In one embodiment, the app can access and present web content to users, such as text and images. Wherein said web content is linked with a corresponding haptic feedback sensation, said feedback is also generated on the device's smart surface so that users can feel it, provided that the actuators present on said smart surface can generate the intended feedback sensations (e.g., textures).
[0032] In one embodiment, the app is displayed on the center stack display device.
[0033] In one embodiment, the haptic feedback sensations can be felt during user tactile interactions with either the display device's smart surface, or the armrest' smart surface.
[0034] The haptic feedback information associated with each web content is to be designed and/or provided by the address hosting said web content. This information is then downloaded by the app, which assesses if the connected devices/surfaces can reproduce the haptic feedback sensations. If yes, the haptic feedback sensations are generated either on all connected and available surfaces, or on those surfaces which the user has previously selected. If not, haptic feedback sensations are not generated.
[0035] It is disclosed a system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
[0036] In an embodiment, the system further can comprise said display for displaying images of patterns or objects, wherein the haptic surface is superimposed on the display.
[0037] In an embodiment, further comprising a seat armrest, wherein the haptic surface is superimposed on the display which is arranged on said armrest.
[0038] In an embodiment, the haptic surface is arranged to communicate wirelessly with the electronic data processor.
[0039] In an embodiment, said display is arranged to communicate wirelessly with the electronic data processor.
[0040] In an embodiment, the automotive setting is an autonomous vehicle and the display is a centre stack display of the autonomous vehicle.
[0041] In an embodiment, the automotive setting is an autonomous vehicle and comprising a secondary display which is a centre stack display of the autonomous vehicle, wherein the electronic data processor is further configured to: provide the web browser online user interface on the secondary display for displaying images of patterns or objects; display the image on the web browser interface on said secondary display.
[0042] In an embodiment, the secondary display comprises:
[0043] a haptic surface comprising actuators for exhibiting tactile textures and superimposed on said secondary display, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; wherein the electronic data processor is further configured to: drive the haptic layers of the secondary display to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed on said secondary display. [0044] In an embodiment, said two displays are arranged to communicate wirelessly with the electronic data processor.
[0045] In an embodiment, the electronic data processor is further configured to select either a frictional or a vibrotactile haptic feedback as a function of the pattern or object to be displayed.
[0046] In an embodiment, the capacitive-based touch sensitive layer is made of translucent plastic comprising LEDs underneath it.
[0047] It is also disclosed a method of operating a system according to any of the embodiments, comprising using said electronic data processor for: provide the web browser online user interface on a display for displaying images of patterns or objects;
[0048] display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
BRI EF DESCRI PTION OF THE DRAWI NGS
[0049] The following figures provide preferred embodiments for illustrating the disclosure and should not be seen as limiting the scope of invention.
[0050] Figure 1: Schematic representation of an embodiment of a home/quick access page layout.
[0051] Figure 2: Schematic representation of an embodiment of a clothing shop website layout.
[0052] Figure 3: Schematic representation of an embodiment of a furniture shop website layout.
[0053] Figure 4: Schematic representation of an embodiment of an interface with a capacitive, haptic feedback on armrest and central display. DETAILED DESCRI PTION
[0054] The present disclosure relates to a system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed. The disclosure also relates to the operation method thereof.
[0055] Figure 1 shows a schematic representation of an embodiment of a home/quick access page layout search or URL navigation field, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents an image preview of a bookmarked webpage, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 103 represents a button to add a new bookmark to the home/quick access page layout, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen.
[0056] Figure 2 shows a schematic representation of an embodiment of a clothing store's item selection webpage, where: 101 represents a preview window showing how the clothing item looks like with the selected fabric material, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents the selection button for users to select which material they want to buy; 103 represents a zoomed in area of the selected fabric material is previewed, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen. [0057] Figure 3 shows a schematic representation of an embodiment of a furniture store's item selection webpage, where: 101 represents a preview window showing how the furniture item looks like with the selected material, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents the selection button for users to select which material they want to buy; 103 represents a zoomed in area of the selected material is previewed, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen.
[0058] Figure 4 shows a schematic representation of an embodiment of a capacitive haptic surface on armrest, where: 11 represents a passenger, 12 represents the seat, 13 represents the armrest with the capacitive, haptic surface, and 14 represents the center stack display that the capacitive, haptic surface of the armrest is in contact with.
[0059] As illustrated in Figure 1, a system can comprise a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
[0060] As illustrated in Figure 2 a schematic representation of an embodiment of a clothing shop website layout.
[0061] As illustrated in Figure 3 a schematic representation of an embodiment of a furniture shop website layout.
[0062] In an embodiment and according to the technical field literature, examples of textures that participants subjectively report feeling during interaction with electrode devices include: wood, leather, paper, painted wall, rubber, and viscous liquid (Bau, Olivier, and Ivan Poupyrev. "REVEL: tactile feedback technology for augmented reality.
ACM Transactions on Graphics (TOG) 31.4 (2012): 1-11).
[0063] Through the use of an electrode, it is possible to modify the friction that exists between the finger and an actuated surface, when there is movement of one or both parts. Through friction, we can manipulate, for example, the feeling of how smooth or rough a surface is. This, however, is normally limited in two respects, namely how smooth the surface on which the finger slides is (because electrode only creates friction, does not remove it, so the least possible friction to create is the natural friction between the surface and the finger); and, also, how much electrical charge can be given to the surface actuated (the higher the electrical charge, the greater the friction generated between the finger and the surface), up to a certain limit, which is depending on the structure of the device itself and its electric system.
[0064] The term "comprising" whenever used in this document is intended to indicate the presence of stated features, integers, steps, components, but not to preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.
[0065] The disclosure should not be seen in any way restricted to the embodiments described and a person with ordinary skill in the art will foresee many possibilities to modifications thereof. The above-described embodiments are combinable. The following claims further set out particular embodiments of the disclosure.
[0066] References
1. KHOSHKAVA, V., CRUZ-HERNANDEZ, J. M., & SHAH, K. (2019). HAPTIC ACTUATOR ASSEMBLY WITH A SPRING PRE-LOAD DEVICE (Patent No. EP3582074A1).
2. Olley, M. F. D., Peshkin, M. A., & Colgate, J. E. (2020). Electronic controller haptic display with simultaneous sensing and actuation (Patent No. US10768749B2).
3. Tesla ©. (n.d.). Software Version 9.0. Retrieved February 17, 2021, from https://www.tesla.com/support/softwarev9?redirect=no.
4. Tesla®. (2018). Introducing Software Version 9.0. https://www.tesla.com/blog/introducing-software-version- 9?redirect=no%3Futm_campaign%3DV9&utm_source=Twitter&utm_medium=soc ial.

Claims

C L A I M S A system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed. The system according to the previous claim further comprising said display for displaying images of patterns or objects, wherein the haptic surface is superimposed on the display. The system according to the previous claim, further comprising a seat armrest, wherein the haptic surface is superimposed on the display which is arranged on said armrest. The system according to any of the previous claims wherein the haptic surface is arranged to communicate wirelessly with the electronic data processor. The system according to the previous claim wherein said display is arranged to communicate wirelessly with the electronic data processor. The system according to any of the claims 1-2 wherein the automotive setting is an autonomous vehicle and the display is a centre stack display of the autonomous vehicle. The system according to any of the previous claims wherein the automotive setting is an autonomous vehicle and comprising a secondary display which is a centre stack display of the autonomous vehicle, wherein the electronic data processor is further configured to: provide the web browser online user interface on the secondary display for displaying images of patterns or objects; display the image on the web browser interface on said secondary display. The system according to the previous claim, wherein the secondary display comprises: a haptic surface comprising actuators for exhibiting tactile textures and superimposed on said secondary display, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; wherein the electronic data processor is further configured to: drive the haptic layers of the secondary display to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed on said secondary display. The system according to claim 7 or 8 wherein said two displays are arranged to communicate wirelessly with the electronic data processor. The system according to according to any of the previous claims, wherein the electronic data processor is further configured to select either a frictional or a vibrotactile haptic feedback as a function of the pattern or object to be displayed. The system according to according to any of the previous claims, wherein the capacitive-based touch sensitive layer is made of translucent plastic comprising LEDs underneath it. Method of operating a system according to any of the claims 1-11, comprising using said electronic data processor for: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.
15
PCT/IB2021/062477 2021-12-29 2021-12-30 System and method for providing a web browser online user interface with haptic feedback for an automotive setting WO2023126664A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP21854695.0A EP4457586A1 (en) 2021-12-29 2021-12-30 System and method for providing a web browser online user interface with haptic feedback for an automotive setting

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
PT117714 2021-12-29
PT11771421 2021-12-29

Publications (1)

Publication Number Publication Date
WO2023126664A1 true WO2023126664A1 (en) 2023-07-06

Family

ID=80225596

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2021/062477 WO2023126664A1 (en) 2021-12-29 2021-12-30 System and method for providing a web browser online user interface with haptic feedback for an automotive setting

Country Status (2)

Country Link
EP (1) EP4457586A1 (en)
WO (1) WO2023126664A1 (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170153702A1 (en) 2015-11-27 2017-06-01 International Business Machines Corporation Providing haptic feedback using context analysis and analytics
CN106887121A (en) * 2017-02-10 2017-06-23 上海斐讯数据通信技术有限公司 A kind of intelligent arrival reminding handrail and based reminding method
US20170192510A1 (en) * 2013-06-11 2017-07-06 Immersion Corporation Systems and Methods for Pressure-Based Haptic Effects
US20180046250A1 (en) 2016-08-09 2018-02-15 Wipro Limited System and method for providing and modulating haptic feedback
EP3467624A1 (en) * 2009-03-12 2019-04-10 Immersion Corporation System and method for interfaces featuring surface-based haptic effects
EP3582074A1 (en) 2018-06-15 2019-12-18 Immersion Corporation Haptic actuator assembly with a spring pre-load device
US20200264772A1 (en) 2017-10-11 2020-08-20 Robert Bosch Gmbh Method for providing haptic feedback to an operator of a touch-sensitive display device
US10768749B2 (en) 2012-05-10 2020-09-08 Northwestern University Electronic controller haptic display with simultaneous sensing and actuation

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3467624A1 (en) * 2009-03-12 2019-04-10 Immersion Corporation System and method for interfaces featuring surface-based haptic effects
US10768749B2 (en) 2012-05-10 2020-09-08 Northwestern University Electronic controller haptic display with simultaneous sensing and actuation
US20170192510A1 (en) * 2013-06-11 2017-07-06 Immersion Corporation Systems and Methods for Pressure-Based Haptic Effects
US20170153702A1 (en) 2015-11-27 2017-06-01 International Business Machines Corporation Providing haptic feedback using context analysis and analytics
US20180046250A1 (en) 2016-08-09 2018-02-15 Wipro Limited System and method for providing and modulating haptic feedback
CN106887121A (en) * 2017-02-10 2017-06-23 上海斐讯数据通信技术有限公司 A kind of intelligent arrival reminding handrail and based reminding method
US20200264772A1 (en) 2017-10-11 2020-08-20 Robert Bosch Gmbh Method for providing haptic feedback to an operator of a touch-sensitive display device
EP3582074A1 (en) 2018-06-15 2019-12-18 Immersion Corporation Haptic actuator assembly with a spring pre-load device

Also Published As

Publication number Publication date
EP4457586A1 (en) 2024-11-06

Similar Documents

Publication Publication Date Title
US10579252B2 (en) Automotive touchscreen with simulated texture for the visually impaired
CN101714026B (en) User interface feedback apparatus, user interface feedback method, and program
US11625145B2 (en) Automotive touchscreen with simulated texture for the visually impaired
Breitschaft et al. A theoretical framework of haptic processing in automotive user interfaces and its implications on design and engineering
CN105045377B (en) Automotive touchscreen control for haptic feedback using simulated texture
Large et al. Feel the noise: Mid-air ultrasound haptics as a novel human-vehicle interaction paradigm
US20140139451A1 (en) Systems and Methods For Providing Mode or State Awareness With Programmable Surface Texture
US9983672B2 (en) Electrostatic haptic actuator and user interface with an electrostatic haptic actuator
US20160342269A1 (en) Tactile sensation providing apparatus and system
Banter Touch screens and touch surfaces are enriched by haptic force‐feedback
CN103838424A (en) System and method for feedforward and feedback with haptic effects
CN106255942A (en) For optimizing the system and method for sense of touch feedback
Riegler et al. Gaze-based interaction with windshield displays for automated driving: Impact of dwell time and feedback design on task performance and subjective workload
US20170097682A1 (en) Tactile sensation data processing apparatus, tactile sensation providing system, and tactile sensation data processing method
Lauber et al. What you see is what you touch: Visualizing touch screen interaction in the head-up display
CN108763391A (en) Questionnaire page surface treatment method and apparatus
JP6700034B2 (en) Techniques for dynamically changing the haptic surface of a haptic controller to convey interactive system information
Berger et al. InShift: A Shifting Infotainment System to Enhance Co-Driver Experience and Collaboration
WO2023126664A1 (en) System and method for providing a web browser online user interface with haptic feedback for an automotive setting
Ecker et al. Visual cues supporting direct touch gesture interaction with in-vehicle information systems
Hashimoto et al. Effect of pseudo-haptic feedback on touchscreens on visual memory during image browsing
Serafin et al. User experience in the us and germany of in-vehicle touch screens with integrated haptic and auditory feedback
WO2023118956A1 (en) Device and method for providing meditation-inducing stimuli for a user in an automotive setting
Beruscha et al. 22‐3: Invited Paper: Deriving User Requirements for Haptic Enhanced Automotive Touch Screen Interaction
Lisseman et al. In-Vehicle Touchscreen Concepts Revisited: Approaches and Possibilities

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21854695

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021854695

Country of ref document: EP

Effective date: 20240729