Defining Patterns for a Conversational Web

The path to a conversational Web started with the early improvements to the linear navigation of screen readers. To lower the complexity of using screen readers, prominent approaches have proposed speech-based extensions that would accept spoken commands as shortcuts to screen reader functionality [6, 55]. Other approaches introduced strategies for content organization and navigation , such as machine-based segmentation of content to enable navigation through semantically related content [12, 13], summarisation of web content [28], and non-visual skimming [3]. Speech optimizations , through the exploration of faster text-to-speech [27] and multiple speech channels [27, 62], try to speed up and enrich navigation by catering to the abilities of BVI users. Web automation approaches also enable the user-defined [11] or automatic [5, 31] creation of macros for repetitive web-browsing actions.

Superimposing Conversational AI (CAI) on existing systems is now gaining momentum [9] and is suggesting new directions for the interaction with digital services. CAI is adopted to grant access to data and services at different levels, from extending GUIs of apps and websites [8, 33, 44], to adding natural-language front ends to Web services, processes [45, 47, 54, 61] and data repositories [17, 42]). On the Web, CAI is often exploited to build pop-up bots [9], i.e., assistants embedded within websites that offer services, such as conversational FAQ and help. However, these solutions do not focus on website navigation and content fruition. Tighter integration between websites and CAI is instead achieved by multi-experience websites, which offer both visual and conversational interfaces on the same content and functionality. For example, a recent approach proposed by Planas et al. [40] speeds up the development of multi-experience Web sites thanks to modeling abstractions and a related domain-specific language. However, the developer still defines the conversational experience by hand as a detached application.

Lately, researchers have proposed approaches that leverage CAI to offer alternative interaction paradigms for the Web. Cambre et al. [16] explore the use of open Internet technologies to build a virtual assistant for the Web implemented as a plugin for the Firefox browser. It enables interactions with browser functionality, and the natural-language expression of complex queries on Google Search for locating specific content items on the Web. Ripa et al. [44] then focus on facilitating the end-user generation of information bots out of website content. The approach is based on Web augmentation and relies on an annotation tool, to allow users to structure the content feeding the bot, and a flow editor to define the order and structure of responses and reading behavior. The end-user is in charge of the conversation design.

Related to all these approaches, some papers promote the idea of a Conversational Web [8] to enable users, especially those challenged by visual interaction paradigms, to express and fulfill their Web browsing goals by engaging in conversations mediated by a CA. One fundamental principle for this paradigm is enabling access to the web’s wealth of services and information, even when websites are not equipped with ad-hoc conversational extensions, and provide (to the possible extent) a homogeneous conversational interface across websites. Progress towards this paradigm mainly refers to technical challenges and directions for tight integration between Web platforms and CAI [20, 39].

The above works bring valuable contributions, yet there is still limited guidance on how to design conversational interactions that can be generated by interpreting the structure of an existing website and transposing content and functionality of the visual Web into conversational experiences. Indeed, most of these approaches outsource the design of conversational interactions to developers or to the final users. Our work wants to take the lessons learned from formative user studies to propose reusable design patterns that can be natively offered by websites.

The literature highlights the potential benefits of conversational paradigms to enable a better user experience for people with disabilities, including BVI people [2]. At the same time, studies using a variety of methodologies have brought a better understanding of the design challenges and the unmet needs of this population when it comes to designing CAs [1, 2, 41]. Prominent challenges relate to the design of input mechanisms, control over the presented information, the interaction modalities, and even privacy when interacting through voice [1, 2, 41]. Branham and Roy [15] suggest that guidelines for the design of voice assistants might contribute to solving these challenges, but argue that current proposals do not properly meet the needs of BVI people. Here, contributions go from general Human-AI interactions guidelines [4], to industry and platform-specific guidelines [59], and recommendations for accessible CAI [34, 52].

Among the efforts toward accessible CAI, Leister et al. [34] analyzed 29 different sources of guidance under disability classifications and assessed the applicability of Web content accessibility guidelines (WCAG 2.1) for CA design. In the end, they derived 23 design considerations for accessible conversational interfaces. Stanley et al. [52] performed a review of 17 different sources from research and practice, and synthesized their findings in 157 recommendations derived from standards, empirical studies, and UX analysis. Similarly, Murad et al. [36] develop (high-level) heuristics for voice UIs inspired by GUI guidelines. While extremely valuable, these accessibility design considerations are general and not targeted to the needs and capabilities of BVI users. More specific guidance comes from a few empirical studies that do leverage the skills and capabilities of blind users [21, 30]. For example, Choi et al. [21] explored speech-rate configurations to meet the exceptional listening abilities of blind users. This study further supports the need for specific design considerations for this population but also highlights that solutions might depend on the type of task, content, and context of use. Still, these considerations do not address Web browsing tasks.

Design patterns for CAI are unexplored to a larger extent. Progress is being made but in specific solutions, such as VERSE [56], which explores the integration of screen reader functionality and voice assistance, with a focus on information search on the Web. VERSE incorporates interesting design ingredients, such as combining breath and depth in search, and supporting navigation commands through different input modalities. However, the actual approach to exploration (of a website) is limited to simple navigation commands specific to Wikipedia. Bouguelia et al. [14] propose patterns for task-oriented chatbots for Web services, and highlight the implications on dialog structure, required abstractions, and supporting infrastructure. Our work elaborates on such implications and adapts them to conversational patterns for Web browsing. The results illustrated in the following sections are grounded on the analysis of the needs and abilities of blind and visually impaired people; however, they still can contribute to a much-needed understanding of how to leverage CAI for accessing the Web.

We started our research in April 2021 with preliminary unstructured interviews carried out with 3 digital-technology experts (average age: 47), all but one blind, who educate and assist BVI people in learning and using assistive technologies. Each interview lasted about 2 hours and, due to COVID-19 restrictions, was conducted remotely through video-conferencing tools. To learn from the experts how to approach the design for BVI people, the whole research team (i.e., all the authors of this paper) attended the remote meeting, with two researchers acting as moderators. The involvement of the experts was significant not only for gathering their personal experience when they approach the Web but also as a proxy to understand the challenges faced by the BVI people they help with their educational activities. The discussion revolved around how screen readers support Web navigation, and to which extent BVI people could accept CAs as an alternative technology. The interviews were video-recorded. The transcripts were analyzed by the two moderators, to perform an inductive thematic analysis [35]. Independently, they double-checked the material; then, iteratively reduced a few variations on the emerging themes (10%), till reaching a complete agreement. The results were finally discussed with the whole team that in the end agreed on three main emerged aspects: (i) the complexity of understanding the website structure and locating information, even for users who are highly experts in screen readers’ usage; (ii) accessibility issues related to the interpretation by screen readers of the page HTML code; (iii) limitations of CAs, first of all, the lack of support for browsing information on the Web, which was also in line with findings from the literature [16, 41]. To deepen and verify these aspects with a larger number of users, we defined and validated with the experts a mixed-method questionnaire including 25 questions, both closed and open-ended, covering three main aspects: experience with Web browsing, experience with screen readers, and experience with CAs.

In May 2021, we advertised the questionnaire in the monthly newsletter of the UICI association and spread it also among BVI people belonging to the other two associations we were in contact with. Table 1 reports representative questions for each addressed aspect². We received a total of 23 responses from users aged between 18 and 65 years;14 were totally blind, and the others had severe visual impairments. Among several aspects, the gathered answers revealed that the Web sites the participants access the most are information-intensive and that accessibility errors in a Web page layout are barriers that they can tolerate. The answers to the open questions, instead, highlighted that more server issues occur for content browsing, especially during the initial orientation within a website’s content.

At the beginning of July, we then conducted semi-structured interviews with 13 users who in the online questionnaire had declared to be frequently engaged with digital services and assistive technologies and had expressed their interest in follow-up activities. They were aged between 18 and 60 years (6 females, average age: 42), and 8 were totally blind. Each interview was moderated by two researchers, lasted about 1 hour, and was held remotely. The discussion was guided by the open-ended questions of the online questionnaire referring to the experience with Web navigation, screen readers, and CAs, but went deeper into the different challenges, probing participant responses, and encouraging them to provide details and clarifications. The questionnaire had highlighted a range of problems and opportunities, but it was the interviews that provided depth on the causes. After integrating the new data with the ones gathered through the questionnaire, the two researchers independently identified the emerging themes through an inductive thematic analysis [35]. The results were discussed in 2 sessions, after which they agreed upon the relevant findings and came to the definition of the preliminary challenges illustrated here below and summarized in Table 1.

Experience with Web browsing. The participants highlighted an intrinsic lack of accessibility for many websites. P15 said: “Even without visuals I would like to experience the Web page without being disappointed every time I need to interact with an unreadable visual element that I can’t understand”. They reported usability problems (e.g., confusing information architecture) that can impact any user, not only BVI people. P24 said: “I cannot detect how the content is structured. The real problem is not incorrectly coded links, but not being able to find the right information”. They also commented on difficulties in identifying the semantics of links. P5 said: “Sometimes I don’t clearly know what I am searching for, and I hope my navigation would be guided by meaningful links”.

Experience with Screen Readers. When using screen readers, learning the website’s structure is a necessary first step. However, the participants commented that this is also the most demanding activity for many websites. P4 observed: “Sometimes I open a site and it takes me 15 mins or more to understand what it is about, where I am, and where I can navigate to”. P22 said “It is a mystery to me where they [links] can take me. I cannot predict my movements on the site”. The participants also complained about the high variability of the content organization, which prevents them from identifying strategies that can work across different websites (P8: “It is challenging to identify standard exploration strategies for different websites”; P21: "I keep going by trial and error. The effort is reduced over time thanks to my experience in browsing, but it remains significant”). A frequent observation came up about the need of identifying the navigational context, i.e., where the users came from and where they could navigate to (P12: “Sometimes I get lost because I cannot remember the page where I came from, and I cannot even return to the home page because I don’t remember the path!”). Almost all the participants reported a high effort to follow the serial reading of the page content (P23: "I usually get distracted because I have to keep in mind everything that the screen reader tells me. Reading one paragraph per time would be more relaxing"). They give up on browsing when they encounter page components with a strong visual connotation, such as online forms, calendars, and pop-ups, which the screen readers cannot interpret properly or even intercept (P26: “Every time I close a pop-up window, I always ask myself what I could have accepted.”).

Experience with CAs. Frequently when talking about CAs, the participants complained about the limited understanding capabilities and the inability to serve many of their information needs (P18: “This technology looks intuitive, but I frequently need to rephrase my request and often it is unable to give me an answer”). The participants also mentioned the lack of a natural conversation flow (P14: “Answers are often disconnected or repetitive, it can’t remember what I asked before and comes up with the same information.”). Given these lacks, participants remarked they don’t trust current CAs. They perceive the technology’s potential but will engage with it only if it shows benefits for easing Web navigation. P16 said: “I would be happy with a CA retrieving Web content with a minimum number of steps, without being overwhelmed with useless embellishments and superfluous text”.

To refine the challenges identified in the previous step, we invited the interviewed users to participate in focus groups. 5 participants responded to our invitation. They were aged between 18 and 48 years (2 females, average: 24 years), two of them were totally blind and the remaining were affected by severe peripheral visual impairments. In the middle of July, we held two in-presence focus groups, the first with 3 participants, and the second with 2 participants. Two researchers moderated the two sessions, in which they observed the users while i) browsing the Web with screen readers and ii) interacting with a CA purposely designed to browse selected Web pages. Since the majority of the previously identified challenges referred to information-seeking and content-exploration issues, Web browsing was the main investigated dimension.

After one week, we organized a 2-hour in-presence co-design session with the same 5 participants, moderated by the same two researchers as in the previous activities. With the aim of identifying further preferences on how to organize the conversation for Web browsing, the participants were asked to define by themselves the conversation for browsing a website of their choice. They choose the website of an escape room that they, especially the youngest, knew very well, yet they claimed it was inaccessible via screen readers. As soon as the first ideas emerged, the researchers helped the participants implement the conversations with DialogFlow⁵ and deployed them on Google Assistant. Participants could run the prototype on their mobile phones and Alexa (see Figure 2). They played with the prototype and iteratively modified it to refine their ideas. The analysis of the proposed solutions highlighted three design dimensions that we discuss in the following.

Navigation. Similar to the design of the CA adopted for the focus group, the participants organized the content hierarchically, to enable a layered exploration, but they also proposed new elements.They worked on providing feedback on the reached status when landing into a new browsing node, and on explaining the possible actions that could be invoked next (P22: "First, I want to understand where I am and what I can do next; then I want the CA to proceed by reading the information on the page”). They reflected on mechanisms to formulate fast-served requests for specific content (P23: “When I open the website, I’d like to ask if it can offer what I am looking for, without necessarily having to scroll through everything”). The users also discussed the benefits of tagging nodes and clustering them into meaningful categories (P24: “It would be interesting to bookmark elements during my navigation, to easily retrieve them and get oriented even in successive browsing sessions!".

Content reading. Similar to the design of the focus-group CA, the participants segmented the page content to quickly locate items of interest (P26: “A serial content reading is OK, but I would like to stop the CA if needed, or to proceed reading only under specific request, for example: "next please!"”). As a new element, for each segment they considered summarization and keyword-highlighting techniques (P25: “It always takes me too long to figure out why this content is the result of my research. Highlighting the main concepts would help me a lot, similar to the preview in Google Search!”).

Bot architecture. The participants confirmed the importance of being able to invoke, at any moment, commands for going back, asking for help, and navigating to landmark pages.

The focus groups and the co-design activities were audio-video recorded. The two researchers moderating these activities individually took notes on significant participants’ behaviors and aloud comments. They transcribed their notes and their post-experience considerations and extended them by integrating the video analyses. Separately, they performed a deductive thematic analysis, clustering codes from in-vivo coding [35] into the themes already identified in the previous phases. The two researchers independently double-checked 70% of this material. The initial reliability value was 80%. After discussion, the researchers agreed upon the differences and reduced the variations till they reached a full agreement.

As further validation, the participants in the previous co-design session were invited to revise the identified insights. 3 of them (1 female, average age: 28 years old) accepted to participate in a 2-hour focus group. To facilitate the discussion, the researchers built a new version of the CA previously defined for browsing Wikipedia pages, now integrating the additional conversational mechanisms that came out during the co-design session.The 3 participants were first asked to interact with the new prototype, then to express their opinions on the relevance of the identified aspects. In the end, with the agreement of the 3 participants, the researchers reached a consensus on the following design dimensions, which expressed in more detail all facets that emerged during the whole process:

Our patterns refer to a model, the Conversation-oriented Navigation Tree (CNT), which suggests a hierarchical organization of the content to be conveyed through conversation - the study participants indeed frequently remarked on this aspect. As illustrated in Figure 3, for each page in a website the CNT represents a hierarchical nesting of both content elements and navigational structures, which recalls the HTML DOM organization but introduces new conversation-oriented elements. Each node in the tree, which we call conversational node, can be a content paragraph, a navigation menu, a link, or any other element in the webpage that can be presented independently of the others and has a role in the exploration of the website content. More specifically, the internal nodes represent aggregated content structures (e.g., “today’s articles” on the Wikipedia Home Page) or navigational indices (e.g., the main navigation menu). The leaf nodes then represent the page’s actual content. This node granularity serves the purpose of building an incremental exploration of webpage content through tree traversals.

The CNT enables conversational interactions with websites in two crucial ways. First, it organizes the website information architecture, independently of the visual layout organization and the presence of accessibility and semantic tags in the DOM definition, making it easier to navigate and consume content by CAs. Second, it contains a minimal set of attributes that are important for the generation of conversational interfaces. Each node is indeed associated with descriptions, which can be already available in the HTML code (e.g., alt-text for images), or purposely generated to enable the conversation (e.g., summaries of text content [32]) thus overcoming the lack of accessibility meta-data that affects many websites. A node might also store keys indexing its content, which are useful for locating and directly access to nodes. Specific nodes can be labeled as conversational landmarks, to have global visibility in the conversation. A node type indicates the role of the specific element and defines the type of operations that can be performed on it (e.g., content, link). Additional attributes can be derived from page content processing and attached to the nodes.

With the above approach, the problem space for enabling conversational interaction with websites is reduced to deriving CNT representations. This can be done by relying on automatic approaches, already explored in the technical literature [7], that build and handle the CNT during the website navigation by exploiting HTML annotations purposely added during the website authoring, or by employing techniques for the automatic segmentation [23] and summarization of webpage content [32].

The CNT organizes information so that, by applying tree-traversing strategies and providing content previews for the traversed nodes, a CA can help the user grasp the overall website organization.

View in the large. As soon as the user enters a website, the interaction with the Home Page must convey the main thematic areas and the main navigational components. Figure 4 illustrates an example of a conversation with the Wikipedia Home Page. On the left side, the CNT represents the hierarchy of the page components. The related conversation, on the right side, starts with a short description of the website content, as extracted from the CNT root node, plus a preview of the main thematic areas linked from the Home Page and the navigation header - these last are the child nodes of the CNT root. The same strategy can be adopted when the user enters the inner areas of the website, by recursively visiting any subtree representing content aggregations. Rollback actions and direct access to landmark nodes (see Section 4.5) further sustain the exploration of the content organization.

Navigational context. Every time the user enters a new navigation node, the CA must offer information on the navigational context. As reported in Figure 4, besides presenting a short description of the reached node, the children nodes can be introduced to help the users understand where they can move to (e.g.:“Here you can read the introduction or follow one of the available links [...]”). Each node can dynamically store a reference to the node the user came from, which can be presented to the users as an option to easily go back (e.g.: “You came from the Home Page”). To keep the conversation fluid, context information can be supplied on demand, i.e., only if the user asks for it (e.g.: “Do you need more information for localizing the node?”).

Link predictability. A preview of the content reachable through a link can help understand a-priori what the target content is about and avoid useless navigation steps. Considering the CNT model, each node representing a link can store a short description of the target content. It can be derived from the HTML link-title tag, when available; the CNT can also provide more contextualized descriptions, extracted from the target CNT node the link points to. The last part of the conversation in Figure 4 illustrates an example.

The hierarchical content organization suggested by the CNT lets the user browse the navigable space by moving down in the hierarchy (in-depth exploration) and exploring the content at each layer (in-breadth exploration). The indexing keys associated with each node also allow the users to locate content by direct queries.

Exploration of thematic areas. As represented in Figure 5, the CA assists the exploration of each thematic area by allowing the users to i) move horizontally across the pages at a level of the content hierarchy (e.g.: “[…] you can find “Name and Symbol”, “Physical Characteristics”, […]”), and ii) move vertically (e.g.: "Go down to “Composition”) to reach inner layers offering further details on a given topic. Considering the CNT representation of a website, this pattern requires strategies for traversing subtrees representing thematic clusters of conversational nodes.

Q&A. The users must be able to formulate punctual, fast-served requests for specific content. If we consider the CNT representation of a website, this requirement implies that each node in the tree is labeled with keys characterizing its content and serving node localization and direct access (e.g.: “Is there anything about “Jupiter’s rings”, in Figure 5).

Bookmarking. The users must be able to bookmark nodes for later access to important pieces of information; this can be beneficial for personalizing the navigation experience and have a positive impact on orientation. The CNT addresses this requirement by storing in each node possible user-generated labels. As illustrated in Figure 5, the CA must then be able to understand and serve user intents to i) label the nodes, and ii) retrieve those nodes in successive browsing sessions (see Section 4.5).

User-defined node clustering. This is a specialization of the bookmarking pattern, which allows the users to group bookmarked nodes in clusters representing thematic areas that are meaningful to them and could help them recall a navigation context (e.g., the category “Astronomy” in Figure 5).

Content reading patterns suggest how to segment, summarize, and index the website content, and how to let the user move through the resulting conversational nodes thanks to dedicated content-reading commands. The original page content must be preserved and should be entirely accessible if the users ask for this option.

Content Segmentation. Page content can be divided into segments that the users can quickly scroll through appropriate commands. Figure 6 reports an example of content reading based on segment scrolling. The CNT can support this pattern if an adequate node granularity is defined. Besides considering the DOM structure of a webpage, vision algorithms can be applied to parse the visual appearance of a webpage and identify segments [22, 23]. A vision-based segmentation preserves the consistency between the visual and the conversational Web - a desire users frequently expressed during formative studies.

Skimming mechanisms. Content summarization can give a preview of a node, for example when a new node is entered or when a link must be traversed and the user wants to know a-priori what can be found in the target node. It can prevent unwanted, or unneeded, content readings and navigations. An example of this pattern is reported in Figure 6, where the CA presents a summary of what can be found in the Jupiter article before reading the entire page. Summaries can be manually defined by the CA designer or automatically generated by summarization techniques [32].

Conversational tag cloud. Inspired by the visual Web, conversational tag clouds can convey the key concepts characterizing a node’s content. Together with summaries, tags can help users assess the relevance of a node before fully reading it. In the example in Figure 6, the CA lists tags that represent the most relevant information within the reached node. In a CNT, each node can store these tags, which can be extracted through summarization techniques [32].

The tree-based organization of conversational nodes and the meta-data stored in each node help manage intents for controlling the conversation and the navigational context, such as:

Figure 7 illustrates examples of conversations where the user invokes control commands.

An important factor for orientation, extensively highlighted by our formative studies, is the homogeneity of browsing strategies across different websites, which is very difficult to achieve given the variability of website designs. Variability in the visual Web might not constitute a problem as visual cues can guide the users to understand how the website is organized. With conversational experiences, it is instead paramount to prioritize consistency across different websites or webpages, to speed up the familiarization with the information structure [46].

Independently of the visual organization of webpages, the CA should provide consistent mechanisms that can help the users retrieve familiar conversational strategies. This can be achieved by adopting a consistent dialogue flow across different websites, with the same mechanisms for exploring and navigating into the available areas, accessing node content, and invoking general commands for conversation control. Figure 8 schematically shows a typical dialog organization based on the patterns illustrated above. The vertical arrow highlights a possible “conversational journey”, with a flow of user intents that go from orientation to navigation and then content reading. This flow is not rigid, as the user’s adoption of the available commands might depend on the specific use and navigational context. However, the CA must support those Web browsing intents, together with control intents (represented by the horizontal arrow) that can be invoked at any conversational step.

The platform then automatically generates the dialogue for serving the user requests. On the client side, a Web browser extension handles the voice-based interaction using libraries for speech-to-text and text-to-speech conversion. Thanks to the integration with an NLP pipeline [43], server-side components: i) interpret the user natural-language request to extract intents and entities, and maps these elements onto one of the available patterns, each pattern being implemented as an intent handler; ii) using a headless browser [49], transform the interpreted requests into navigation actions to keep the navigation status updated, and iii) automatically generate natural-language responses based on the activated intent handlers. The technical feasibility, the performance of the NLP-based generative techniques for handling the dialog, and the optimization strategies to achieve low response times are discussed in [7].

Current literature offers a valuable general framework for designing accessible conversational experiences [34, 37, 52] but the focus on Web browsing is limited. Our HCD process specifically reflected on Web browsing practices and challenges faced by BVI, and involved the observation, analysis, and co-design of conversational experiences across different informational websites. We run the evaluation on Wikipedia for having a quality representative baseline (i.e., a well-structured and accessible website) with which the users had prior experience, so that to gather user feedback on Web browsing and navigation through conversation, not on other factors that might derive from an unnecessary complexity in the website structure and accessibility problems. Thanks to the specific attention posed on content browsing mechanisms, also in comparison with other prominent solutions addressing Wikipedia, such as VERSE [56], our proposal goes beyond basic navigation commands and proposes articulated patterns that seem to respond to the Web browsing challenges identified during the HCD process.

The preliminary validation suggests that the devised patterns can effectively support Web browsing while addressing some of the most prominent challenges faced by BVI users today. Participants perceived that the conversational paradigm enabled a more natural interactions, allowing them to more directly express their Web browsing requests without having to enact complex user workflows operating a screen reader. They perceived a reduced cognitive effort, offloading the need for memorizing the structure of websites. They acknowledged being facilitated in the creation of a mental map, without having to explore the entire content of websites, and that the dialog flow lowered the barriers to learn the navigational approach and capabilities. While these results are preliminary, the response was positive and provides a foundation for larger-scale studies on the impact of the conversational paradigm and patterns in Web browsing scenarios.

Likewise, users expressed diversified needs for content reading, especially concerning in-text link detection: from communicating explicitly the presence of a link (P2: “Personally, I would like to hear the word “link” every time, but I am aware that in Wikipedia this could be irritating for someone”), to providing recognizable earcons (P1: “I prefer a nice background sound instead of listening to an interrupted dialogue”), or even to disabling any mechanism to prevent annoying repetitions (P4: “I first read the entire text and then proceed by searching for links I could be interested in”).

This variability in the participants’ preferences for voice-based content presentation lets us think that the configurability of conversational elements is the key to improving the conversational experience for any user, and will therefore be the object of further studies and design activities.

Direct queries vs exploration. Recent studies have identified undergoing changes in mental structures for file browsing and storing among young people [19]. Years of googling and social network browsing have changed how information is perceived and retrieved. During the validation phase, participants tried indeed to access information with direct queries, both for getting oriented and exploring the navigation options. Even while consuming the content of single pages, direct requests were the most used (“What about x?”; “Where can I find y?”). Contrary to our expectations, the layered exploration from the home page to the inner nodes, which emerged as a need during the formative studies, was then mostly abandoned. Future investigations could examine whether this trend is confirmed with a more extensive and diversified set of users.

Multi-experience paradigms. Participants with visual capabilities, who use the visual segmentation of the text and the length of paragraphs to orient themselves within a webpage, pointed to a deeper synergy among visual and conversation paradigms (P23: “I can also orient myself "by sight" while using ConWeb! I cannot read the text, but I identify the correspondence between text reading and the “visual shape" of the paragraphs."). In line with previous works [40], one interesting direction to meet the diversity of needs could therefore be a mixed paradigm that, besides leveraging the conversation, could also offer an integration of visual and conversational access. As reported in the literature this would be beneficial for other classes of users, for example, older adults [29]. Integration with screen readers. For most of the participants, the opportunity of naturally conversing with a webpage was thrilling, yet some participants expressed doubts. P22, who in the final phases of the study expressed great enthusiasm for the new paradigm, in the initial focus groups had observed: "Screen readers will always be my preference as assistive technology for their "passive" nature as I would not be forced to have a verbal interaction". Also in line with previous work [56], this suggests that the next studies should put effort into discovering new synergies among existing assistive technologies (such as screen readers) and the conversational paradigm, to give the users a chance to choose the most adequate one depending on their tasks and the context of use. Since the identified patterns and the logic governing our technological platform do not prescribe any specific input and output mode, the integration is feasible and can be achieved through adequate communication and synchronization mechanisms, with the conversational paradigm becoming an orchestrator.

Patterns coverage. Our studies focused on conversational experiences often inspired by informational websites with a regular and consistent organization, e.g., Wikipedia. If, on the one hand, this allowed us to addressmany aspects of conversational Web browsing, on the other hand, it could constitute a limitation for the generalization of the approach to other classes of websites.The need to address the presentation of dynamic components already emerged during the conducted studies: when using the screen reader to interact with a train-reservation website, the participants were not able to select the departure date on a calendar component (P25: “These letters spoken by the screen reader I believe are the days’ initials, but it took me a bit to figure out”; P26: “We had a lot of troubles inserting the departure time since we had to select it from a calendar visualization”). So far, we have given priority to the design of patterns focusing on textual content within Web pages but our current work is already devising solutions for intercepting and presenting dynamic page components. We will also extend the evaluation to the navigation of more complex websites. However, we are confident that the defined patterns can still provide a solid basis for organizing conversational access.

CNT management. On the technical side, our approach relies on the construction of the CNT model, which is built automatically every time a webpage is accessed. The dialog is also automatically built depending on the created CNT instance. In our current prototype, these mechanisms work perfectly if the website HTML is augmented with proper tags (i.e., for websites natively instrumented to be accessed through ConWeb). They are highly accurate, even in the absence of ad-hoc tagging, when websites have a regular structure, while the page interpretation may fail with highly dynamic websites and dynamic components. This might hinder the adoption of ConWeb, and that is why further work will focus on making page interpretation more robust.

Diversity of study participants and sample size. One limitation of the studies was the age of the participants who were almost all young adults. The recruitment for these phases was done on a volunteering basis and the young adults were the most enthusiastic. While on one side this can hinder the representativeness of the insights for the regular user population, on the other side user comments pinpointed the youngest users as the adequate target. Older participants might be too accustomed to screen readers to appreciate or even show an attitude toward new technology. For example, P17 said: “Learning screen reader technology was a slow and difficult process. At my age [64 years old] I do not want to spend even more time trying something new because what I use already works enough for me!”. Also, participants were particularly hard to find for the study, resulting in a limited sample size. Future studies can specifically focus on assessing the attitude toward the new proposed technology among users of different ages, thus enhancing diversity and improving the robustness of the results. Gender-related issues could also be investigated, since out of the 26 participants in our studies only 6 were female. Finally, the study participants were very interested and motivated to adopt the assessed technology. Diversity in relation to the participants’ experience with screen readers and digital technology should be pursued.