Abstract
Bridge inspectors work for the safety of our infrastructure and mobility. In regular intervals, they conduct structural inspections – a manual task with a long-lasting and firmly normed analogue tradition. In our collaborative research project, we developed Mixed Reality (MR) and Virtual Reality (VR) prototypes to support that work digitally. We propose a mixed analogue and digital workflow using Building Information Modeling (BIM) data that can be ready-to-hand for bridge inspectors during their work on-site at a bridge. In this paper, we describe the system and the evaluation results of our final MR demonstrator at a autobahn-bridge in Germany. We identified a need for a digital MR tool to support the bridge inspection in-situ. In general, this matches with the trend to bring the computer-supported office-work out into the real world. However, there are also challenges to consider, like lacking BIM data for existing bridges and structures, appropriate user-interfaces in this new application domain, or the need to adopt norms and guidelines for public tender. We argue to consider a user-centered design approach for future developments to best profit from the bridge inspectors’, as well as the MR- and CSCW-researchers expertise, and ultimately increase the acceptance of the developed information systems.
1 Introduction
Roads and bridges are crucial parts of our transport infrastructure and thus essential for our mobility at large. Regular structural inspections are needed to ensure their safety. In Germany, there are 39,500 bridges in the net of national highways alone. Further bridges are in the responsibility of federal states and local authorities. Bridge inspectors are working collaboratively on their examination – a manual task with lots of hammering, currently widely documented using pen, paper, and digital photos on-site and using PC-based software for structured entry back in the office.
While we do not claim to digitalize the inspection process itself, we believe that the documentation process on-site can benefit from digital tools, including Mixed Reality (MR) views. For the inspection itself, the manual character combined with the knowledge and tradition of the bridge inspector is inevitable to identify all possible damages that may occur. To achieve that support for bridge inspectors, we developed a prototypical MR application demonstrator that employs Building Information Modeling (BIM) data. BIM is a method to accompany the lifecycle of buildings and structures.
This method is mandatory for new public construction projects in several countries across the world, including e. g. UK and Germany (currently “only” for all infrastructure projects, public buildings to follow). But also in the private sector, it is getting adopted more widespread. The main advantage of BIM is the digital data exchange between the parties involved throughout the process and earlier detection of conflicts before construction, resulting in a cost reduction through the frontloading of certain activities. BIM offers advantages in the maintenance of buildings: The as-built-BIM-model is understood to contain every information of the structure, such as the current condition (damages, structural changes, and so on). We see a great potential in combining BIM with the possibilities of MR visualizations and digital documentation, which is ground in CSCW theory, especially for bridge inspections.
This paper reports on the final evaluation of a MR prototype that was iteratively built over 18 months to digitally support bridge inspectors using BIM data on site. Our evaluation points out the potential of our approach while also demonstrating its limitations as well as ideas for further improvements.
The remainder of this paper is structured as follows: First, we point out related work. Then, we describe the processes of a digital structural inspection, followed by a system overview and the evaluation design. Hereafter, we present and discuss our results. Finally, we provide a short conclusion and outlook.
2 Related Work
Van Nederveen and Tolman introduced the term “Building Information Modeling”, short BIM [33]. BIM is a method that covers the entire lifecycle of a building or a structure, including planning, construction, operation, maintenance, renovation, and tear-down [7]. Its predecessors date from the mid of the last century: In the 1950s, the first systems’ evolution started [32]. Sutherland developed “Sketchpad” in 1964 as one of the first Computer-Aided Design (CAD) systems [31]. Autodesk’s whitepaper in 2003 [1] initiated a more widespread usage of the term BIM and the method itself. Nowadays, BIM is mandatory for public sector projects in various countries worldwide, like Singapore [2], Great Britain [20], or Germany [15].
Characteristically for BIM are several central digital models (e. g. for different trades) that contain information on the geometry of a building and additional data like employed materials or further information on producers etc. All of the models can be combined into one coordination model. Bew and Richards proposed several levels of detail to define the quality of such a model [4]. Regarding the data exchange, file formats play a significant role. The non-profit organization buildingSMART (https://www.buildingsmart.org) addresses standardization of such file formats. For example, they standardized the Industry Foundation Classes or short IFC file format available for data exchange. For bridges, there is an extension to the standard called IFC-Bridge. And for collaboration and issue tracking, the BIM Collaboration Format (BCF) is notable. One can visualize BIM data using Mixed or Virtual Reality. Brooks’s “Walkthrough” system presented a visualization system for building CAD data [9]. Since then, hard- and software has evolved, and the devices became smaller while getting more powerful. Another example for BIM data visualization is “ARTHUR,” a research project that focussed on architecture and urban planning. The main component was a roundtable enriched by building and city district views on Head-Mounted Displays [8]. Likewise, but outdoors, the Tinmith-Metro system presented a wearable AR system for use in a city context that also supported in-situ 3D modelling [25]. “VIDENTE,” a research project carried out by the University of Graz, developed a portable system that displayed information on pipes and buildings in-situ. Therefore, Ultra Mobile PCs had to be extended by additional sensors, like GPS [29].
Other more recent examples include “Corsican twin”, a system that allows to author Augmented Reality (AR) visualizations in Virtual Reality (VR). It uses BIM and CAD models as one use case [26]. Garbett et al. built a collaborative multi-user BIM-AR system to support the design and construction [17]. Their approach is similar to ours, employing marker-based tracking and evaluating their system with construction practitioners, although they were targeting other stages in the construction process (design and construction) than we did (maintenance and inspection). Hansen and Kjems focus on the interaction challenges with infrastructure data in the field, demonstrating two prototypes for a bridge and a highway in Denmark [18]. However, they evaluated their systems only on a technical level and did not focus on the user perspective. Cuong et al. implemented a system to visualize BIM data of bridges on a HoloLens inside the office [12]. They focussed on the data model and did not include an in-situ inspection.
Industrial applications visualizing and interacting with BIM data include products from companies such as Trimble (https://mixedreality.trimble.com/), vGIS (https://www.vgis.io/), or GammaAR (https://gamma-ar.com/). There are also examples of VR visualizations for bridges [23] or highway construction [36]. Further examples for MR applications in the construction domain are included in [11], [34], [35]. Besides that, researchers also proposed systems for educational purposes, like [16], [21], [28]. The previously presented work has in common that they do not focus on the damage acquisition for bridge inspections. Furthermore, their focus lies more on the visualization of BIM data or building geometry but not on the interaction with that data. With the prototype presented in this paper, we aim to provide a system that respects the requirements and needs of bridge inspectors to support their workflows digitally, thus following a more user-centered and not technology-driven approach.
For this, the combination of Computer Supported Cooperative Work (CSCW) and Mixed Reality (MR) can be beneficial, as pointed out by Ens et al. As state by them, MR can benefit from the CSCW knowledge, and CSCW concepts can be adopted and applied for MR applications [14]. Especially for BIM, a detailed role and access right management is mandatory, e. g. for the Common Data Environment (CDE), which has previously been reported in early groupware concepts from CSCW research [3]. And for future prototypical developments, tighter collaboration between the users seems evident and reasonable. The interaction with the environment may create places following Harrison and Dourish’s definition [19] that different users can share to experience the extended reality together. In our prototype, this is realized through data synchronization so that it may be possible to collaborate asynchronously with a remote expert in the office if a connection was available. Yet, we did not examine this scenario in our user study, as network coverage in bridges is currently generally not available.
We understand MR as an umbrella term for AR and Augmented Virtuality (AV), according to Milgram et al. [22]. They proposed to divide the spectrum between Reality and Virtuality (or VR, Virtual Reality) into the mixed forms AR and AV, with gradual differences in the amount of the real picture one can see. MR was proposed as an umbrella term for all mixed forms of the spectrum. Billinghurst clarified recently that VR is not included into MR [6]. Hence, whenever we refer to MR we see a mixed form, in our case mostly AR, e. g. on a tablet; whenever we refer to VR, we see a purely virtual environment on a PC or on an HMD.
The structural inspection working area has a long-lasting tradition. There exist several rules, regulations, and norms to respect when inspecting structures and bridges in particular. In Germany, the “Deutsche Industrie Norm” DIN 1076 norm contains guidelines for bridge inspection performances. Furthermore, it covers the required documents for monitoring and examination. Besides that, other laws and regulations apply, depending on the bridge’s ownership – be it on the national, the federal-state, or the local level.
3 Motivation
The area of bridge inspections has a long-lasting analogue tradition. Nowadays, bridge inspectors are mostly using pen and paper together with digital cameras to protocol their inspections. This leads to media disruption, as they need to digitalize their findings and damage drawings when they are back in the office. Furthermore, they need to bring together the acquired damages with the pictures taken. In the end, they are required to create an inspection report including the acquired damages and a condition assessment on a grading scale. While we will not concentrate on the condition assessment including the rating of damages, as there already exist software tools that provide a solution for that, we focus on the digitalization of the entire process, as we see potential in developing the damage acquisition process further in a seamless, media-disruption-free, manner.
We argue that user participation and an applied research course of action are crucial for the evolution of the field of digital bridge inspections. Particularly, if such tools should be introduced for the actual bridge inspection task in the future. Hence, this study aims to evaluate the needs and the feedback of bridge inspectors on a prototype we developed in our research project.
4 System Design
We developed a MR and a VR prototype. In this evaluation paper, we are focusing on the MR prototype that is mainly meant to be used on-site at and in the bridge. We found that the bridge inspectors that were part of our evaluation emphasized on the importance of on-site MR support instead of a VR-supported preparation or debriefing of their inspection. For that reason, we decided to focus on the in-situ support tool. The system design is similar for both prototypes, though. We selected Unity as it provides us with support for desktop, HMD, and mobile environments. Bille et al. describe workflows with Unity for BIM data visualizations [5], supporting our choice. Additionally, we use the integrated Unity ARFoundation library and the underlying ARKit for realizing our MR prototype.
Discussion on a damage inside the box girder using the prototype. One can see the input form on the tablet screen with a damage picture and a blue damage sphere (in a later version changed to red) indicating a damage in the live camera image.
Regarding the employed hardware, we chose an iPad Pro for the MR system. Another option would have been to use smart glasses, like the Microsoft HoloLens. However, the expert bridge inspectors preferred the tablet option after comparing both in a kick-off workshop. Their main reasons were the devices’ form factor, the more intuitive handling of the tablet (they already used ruggedized Windows tablets in their work), the HoloLens not being practical for being used in a bridge, and the cost. At the time of the comparison, a HoloLens costed about three times more than an iPad Pro. The second HoloLens version was not available on the market, yet. Other points of criticism regarding smart glasses were the limited field of view and the gesture input system. A tablet, in contrast, allows for shared experiences and thus supports collaboration between multiple users (cf. Figure 1), as other studies show [24], [27].
Implementation of the test order with the digital structure test.
In our research project, we first conducted expert interviews with nine field experts to identify their requirements. Furthermore, they helped us to define the process of a digitally supported bridge inspection. That process – defining the basic steps for the usage of our prototype – is divided into three main processes, which cover the period from the commissioning to the fulfilment of the order for a bridge inspection, as shown in Figure 2. After the order has been placed, the first step is to prepare the inspection in the bridge using a computer or tablet. In this process, the user synchronizes the necessary data from a CDE with the terminal device. The synchronization provides the user with the current inspection report with their located damage and the 3D model. In addition, the user receives further information about the structure, such as the structure book, structural design and past inspection reports. Through a previous virtual view of the structure, the user obtains an overview of existing damage and can optimize his process for the inspection.
The second process phase deals with the execution of the structural inspection of the bridge, where the focus is on the damage investigation and recording. A photo of the damage is taken by the user and associated to a location. The damage can thus be located in the 3D model. After the photo has been taken, a damage recording is made, as part of which the user selects a type of damage from a drop-down list (cf. Figure 6). Furthermore, additional information about the damage, such as dimensions, can be stored. This step is repeated for all damages in the bridge, as decided by the bridge inspector. The recorded information is stored in a BIM Collaboration Format (BCF) file in a predefined scheme with the photo and the coordinates and synced with the CDE once back online or in the office.
In the third main process, the post-processing of the structural inspection takes place in the office. In this process, the user can again gain an insight into the bridge and the existing damage by means of VR/AR and transfer their findings into other systems for follow-up actions and logging. In the current status quo, a lot more preparation and work afterwards is needed, as bridge inspections are widely carried out using pen and paper and the results are transferred into a report later. Thus, there is a media disruption that may be erroneous and time-consuming. Hence, bridge inspectors seek for a digital support solution.
After identifying the process, we tested internal prototypes with bridge inspectors from our projects in several workshops on site despite the ongoing Covid-19 pandemic, using a second test bridge as an example. The collected feedback from the bridge inspectors and the observed technical challenges and difficulties feed back into the implementation process, leading to an inherently iterative approach.
Screenshot of an early prototype test in the office showing a section of a bridge with annotation indicating the German terms. When clicking on a term, one gets to read the metadata on a screen-filling overlay.
The model preparation was an important step for us, as we wanted to ensure a certain amount of metadata as well as a reasonable level of detail. We were provided with a bridge model for the bridge of our evaluation and created a model for our pre-test bridge by ourselves using state of the art modelling software, in our case Autodesk Revit. First, these models were tested inside the office (c. f. Figure 3) and then carried out to the real structures. We had to reduce the amount of detail to get the models loaded dynamically into our application and provide a fluent experience, especially on the iPad. Thus, we pursued an indirect workflow, converting and optimizing the original IFC file to FBX (Filmbox) before importing it into our application. We found that reduction as non-problematic, as the model is used as a rough orientation afterwards or is even hidden by the user to concentrate on the damage recording process.
Our MR prototype splits into the main menu scene, an AR view scene for in-situ use, and an AR scene for the office. With the latter, one can place the model on the office floor and view the geometry, the building data, and the damage data. The data is stored in a CDE and synchronized on the tablet through a Representational State Transfer (REST) interface, as shown in Figure 4. We use the BSCW (Basic Support for Collaborative Work, https://www.bscw.de/en/) groupware system as the public administration for federal highways in Germany already utilizes that software. Nevertheless, our prototype requires a newer version because only newer BSCW versions include the REST interface functionality.
Overview of the system architecture with focus on the synchronization with the Common Data Environment (CDE).
Screenshot of our MR prototype with half-transparent model overlay showing also hidden structures or structures not visible from the location like tendons.
The bridge inspector prepares the tablet by synchronizing the model and all associated data. To localize the tablet and link the digital model to the real world, we apply marker tracking and the iPad’s SLAM capabilities. The iPad automatically scans its environment and if a loop closure or a marker is detected, it corrects the camera position. The user can manually adjust the model position and orientation manually by pressing the appropriate buttons hidden in the menu if necessary. Furthermore, we use iPad’s SLAM also for tools like measuring distances (cf. Figure 6). The user can change the opacity of the virtual model using a slider and even select to disable it entirely. He can add a damage entry by pressing a button (top right, Figure 5). The tablet then takes a picture automatically and records the current position and orientation of the camera. He may then enter the details of the damage using a form that reflects exemplarily the logic of the current state-of-the-art PC-based tool “SIB-Bauwerke” (cf. Figure 6). In the given example, the bridge inspector records a crack in the concrete including the information that it is only at one spot with a total length of 0.58 meters. Later, he can add the width of the crack as a second dimension using his analogue crack width card to fully describe the crack and rate its importance later. The “SOS” dummy-button (Figure 5 and Figure 6 bottom left) only displays an info text and was not evaluated.
Screenshot of damage recording with integrated measurement tool and damage input form following a structure well-known to the bridge inspectors on a real thin crack inside the bridge’s box girder. A red sphere indicates the damage position.
5 Evaluation
Testing took place between July 5th & 8th 2021 at the “Intelligent Bridge” at the autobahn-crossing A3/A9 near Nuremberg in Germany, which is a test-site for autobahn digitalization projects conceived by the Federal Highway Research Institute (BASt) and operated by the Federal “Autobahn GmbH des Bundes”.
5.1 Evaluation Design
We tested with 17 domain-experts from different regions of Germany (Bavaria, Hamburg, NRW), all of which were male civil engineers, 7 of which explicitly mentioned to be active bridge inspectors. Nonetheless, all participants were entrusted with bridge inspections, meaning that those participants who explained to be not active are still in decision-making positions or positions responsible for bridge inspections. All participants were 30+ years old, with 6 being between 30 and 39, 1 between 40 and 49, 7 between 50 and 59 and 3 over 60 years old.
Overview of the general evaluation structure.
The evaluation was carried out in three steps (cf. Figure 7). First, the participants were briefed about the research project and the demonstrator. In the following workshop, the participants were given the opportunity to test the demonstrator independently and then to rate it by means of open and closed questions in a computer-based survey. In the final debriefing, expectations and goals were discussed and noted down by the researchers.
The briefing included a short introduction of the project participants. Afterwards, the project was explained to the users in more detail. They were introduced to the workshop procedure and the goals of the research project. In the further course, the participants were introduced to the functions of the demonstrator.
In the actual hands-on workshop, the participants were able to test the demonstrator on their own in a defined part of the bridge (one station outdoors, one station next to the ladder to climb into the box girder and one station inside the box girder with several damages at each station). We installed four markers inside the bridge: One next to the entrance door to cover the area outdoors below the bridge, one next to the climbing ladder at an intermediate level, and two inside the box girder (one at the start and one at the end of the bridge sector, with a distance of roughly 50 meters between them). The main focus was on how well the participants could handle the functions, especially the damage recording. During the test run, the participants were constantly accompanied by a project participant in order to stimulate a joint exchange of ideas directly via the think-aloud method. These are particularly important for the evaluation. Each participant was asked to acquire at least one damage and had the opportunity to test the system for at least 5 to 15 minutes.
The scope of functions included the localization of damage positions, photo documentation of damage, the damage description, damage examination (measuring) and damage marking. After the practical application was completed, the subsequent survey on the demonstrator was conducted. The survey was divided into four groups: General, demonstrator, ideas, demographics & other.
Participants were asked questions about the general demonstration to find out if the presentation of the demonstrator was sufficient. The questions about the demonstrator asked about the functionality, performance, and equipment. In the area of ideas, the participants were asked to describe what suggestions they had for expanding the demonstrator. In the field of demographics and others, the participants were mainly asked questions about themselves.
In the debriefing, there was a final open exchange with the participants. Here, special attention was paid to the extent to which the building inspection and possible methods for digitalization can be brought together. However, it also became apparent that the participants had some suggestions for improvement, which had already been described in the survey. We will now describe the results of our evaluation.
5.2 Quantitative Results
An initial question asked the participant to rate the just perceived demonstrator on the common German school grade between 1 (Very Good) and 5 (Lacking). The demonstrator was rated on average 2.18 (SD = 1.13), so as “Good”.
We further asked a range of questions with a rating scale of 1 (Very Bad) to 5 (Very Good). The participants assessed themselves on this scale for overall technical knowledge on average as 3.58 (SD = 1.07), knowledge of bridge inspection (3.88, SD = 0.93), knowledge of AR/MR (2.36, SD = 0.92), and knowledge of VR (2.09, SD = 1.38). For AR/MR/VR, 6 participants did not, and presumably could not, answer the question.
All participants tested the core functionalities damage recording and location functionalities. For time reasons, only 13 could test the additional measure tool, and 11 the additional virtual annotation tool. When asked for the functional range of the demonstrator, participants rated it as 3.94 (SD = 0.85).
The precision of the overlay of the virtual bridge model on the real bridge was rated as 3.06 (SD = 1.06). The likelihood of using a digital tool like the demonstrator for bridge inspection work instead of paper was rated rather highly as 4 (SD = 1.22).
5.3 Qualitative Results
We asked our domain-experts a range of open-ended questions that are summarized below.
When asked about the general advantages of the approach, participants mentioned an increased precision in locating damages and a huge time-saving potential, as using a digital tool during damage recording would reduce the time required for post-processing the survey. Traditionally, this often means transferring results from analogue paper-notes to the commonly used software SIB-Bauwerke back in the office. Fully digital damage logging instead was seen as more consistent, less error-prone, and as “documenting without gaps”. It was also seen as providing a better overview of the damage history, aiding in orienting at and in the bridges in general, and when searching for damages. Furthermore, a tablet-based tool would deprecate the need of having to carry a separate camera and void the necessity to work with paper in most cases.
Disadvantages of the approach were mostly seen outside the scope of the demonstrator at hand. One participant found it too time-consuming to work with the demonstrator, and another mentioned it did not provide enough orientation, yet. A danger of information overload was mentioned which could ask for too much attention to the screen, forcing the bridge inspection to concentrate on logged damages and thus distract him from finding new ones. Further potential issues were seen with the currently missing compatibility to existing software, lack of ruggedized hardware (against dropping, dirt, and humidity), and thus danger of data-loss and inability to work in case of hardware damages. Also, using the tablet with certain gloves was not possible.
Asked about the option for a virtual de-briefing and post-processing of the survey (not tested in this study), 14 out of 17 participants would find this useful. They mentioned the following advantages: being able to recap on specific, unclear damages, improve damage positioning and descriptions, speak to colleagues and get a second opinion, being able to visualize and post-process important damages.
Asked about what they esp. liked about the demonstrator, participants highlighted direct damage positioning and categorization on the spot, combined with documenting the damage with photos in a structured way, i. e., attaching it to the right dataset automatically without having to sift through cluttered photo-folders afterwards and manually uploading them. A further identified relief was being able to see and add damages attached to the virtual model superimposed into their views of reality as they make their way through the bridge. This was said to reduce the need to sketch damages on paper and thus speed up post-processing. Having everything stored in one place was seen as a big advantage, as it replaces the traditional and frequent search through cluttered file-system-folders and archived notes. Further points to like were good handling and size of the tablet, measuring on the spot, no need to carry paper plans, cleaner notes due to lack of handwriting, and the potential to remove the need for a post-processing step. Overall, the demonstrator was rated as very hands-on and close to the actual requirements of the bridge inspectors; nicely showing the potentials and weaknesses of such an approach.
Some dislikes about the demonstrator included a lack of precision in positioning, while also admitting it to be a good state of the art already. Again, a link to the existing software in bridge inspection was mentioned, or maybe the fusion of the existing interfaces of SIB-Bauwerke and the demonstrator into something new. Some disliked the handling of photos, such as the need for a photo when recording a damage, or the missing option to delete photos. The user-interface reportedly needs some usability improvements, as e. g. some options were a bit tedious to use, such as the additional measure tool. Also, an option to group similar damages was reportedly missing.
Some participants reported about their expectations as follows: simplification of recording damages (with sketches), easing work due to not having to carry paper-plans around, and automatic optical crack-recognition such as in tunnel-scans. One participant wrote: “I expected a software that would act as a digital tool to the bridge inspector and support him at recording, describing and logging damages based on pictures.”
71 % of the participants thought it would be useful to include the relevant norms and guidelines, as well a checklists, catalogues, and handbooks into the program for reference. Regarding the norms themselves, 41 % of participants saw a need for adapting them with regards to using digital tools in the process of bridge inspections. The norm-committee (DIN NA 005-57-04 AA) should pay particular attention to the latest digital developments. The required software and hardware need to be specified for public tender. This can also be done through inclusion in guidelines and norms. The digital work-processes, file-formats, and interfaces must be specified and incorporated into policies. This must also include compensation for the bridge inspectors, as well as the software developers. Asked about using the device (tablet) itself, it was rated very favorably. One participant emphasized that the use of tablets is very useful and necessary and that such technologies should be considered in the standards. Bridge inspection could be made more timesaving, economical and comprehensible through the use of such new technologies and knowledge. In the long run, he said, this method of data collection, management and processing also offers benefits to the structure itself, as necessary maintenance measures can be arranged much more quickly and efficiently. Another wrote: “Very useful! Working with vast numbers of paper, sketches, notes, unlinked photos, etc. in post-processing, as well as the tedious (double) data-entry and creation of digital versions of the paper-sketches in the office is no longer timely.”
When asked about further devices for supporting bridge inspection workflows, ten participants could imagine a need. This included laser scanners, Dictaphones, and drones, as well as a link to CAD-software.
Further ideas for improvements and additional functionality included the following aspects.
Regarding Usability, UI and program interfaces, participants expressed the following wishes. First, participants wished to combine the demonstrator with existing standard software or provide ways for a smooth data-exchange. Furthermore, they remarked that classifications and texts could be harmonized with existing standards. Further meta-data about the bridge (building number, direction of testing, etc.) would be beneficial. In general, they suggested specific user interface (UI) improvements for streamlining and error-reduction, e. g., selection-options based on current constructional element, or better user-guidance through the crack-recording process through reordering of steps and better default values, prioritize dropdowns over buttons, units over values, location over images, input a series of values for certain damages, and an option to highlight important damages. Also, it would be helpful to provide dynamically filtered views, e. g., to only show data needed for a current task, or segment the bridge into different parts (better overview). Quite similarly, they would prefer to show details on demand, e. g., element material (concrete, reinforcement, wires, etc.), contractors, or warranties. Regarding the input options, pen-based hand-writing entry and recognition is seen as helpful. Users could benefit from zoom-options for photos to look more closely into captured damages and a history for damages that have been fixed.
For localization, they desire a higher precision with damage localization. They see that more markers in the bridge may aid localization (which could be combined with other tech like RFID). Additionally, they would like a support for modification of data for clean-up, e. g., improving locations by moving the spheres.
To ease preparation and debriefing of structural inspections, they think of linking with following maintenance work-steps, e. g., support generation of tender specifications and plans or a navigation to the bridge.
Besides that, they had the following ideas for further developments and functionalities. A semi-automatic crack-recognition by the tool after marking them could help to classify and identify the damage. Damage recording for more structures and parts should be supported in the future. Also, collecting and clustering/layering of similar damages throughout the bridge could be beneficial to them. For that, an investigation of links to artificial intelligence for automatic recognition of damages could be reasonable. An awareness functionality, incl. network availability, position and orientation in the bridge, and emergency could increase their work safety. Regarding the measurement tool, some participants wished a support for area measurement. Furthermore, linking in monitored sensor-data (forces, temperature, etc.) and other measuring tools could be beneficial in the future if bridges or handicraft tools have the corresponding sensors. Besides that, copying existing damages as a template for new ones seems to be an option for some participants. They also can imagine using size and colour of spheres for a more meaningful representation of damage types, location, and status, i. e., to avoid double-tests and show who tested it and when. Additionally, they wish that there is no limit for uploaded data, e. g., photo-size, like it is the case in the current bridge inspection software “SIB-Bauwerke”. Support for putting up traffic signs is mentioned by a participant, in a way that the plan for traffic signs is integrated into the BIM model. It could be also helpful to assess the damage directly in the tool. An opportunity to generate damage reports from the tool could help to reduce the debriefing of a bridge inspection. Regarding data-safety, a participant suggested to e. g., make automatic local copies when in offline-mode to prevent data loss.
6 Discussion
In the following, we will discuss our results clustered into the topics Usability, UI, and program interfaces, localization, model quality, preparation and debriefing, and additional functionalities and ideas. It is to mention that the rating of the demonstrator was confusing for the participants, as we inverted the rating scales for that question. That is a lessons-learned for us: For upcoming evaluations, we will keep the rating system of each question equal.
6.1 Usability, UI, and Program Interfaces
Overall, the participants agreed with our device selection. For application in practice, battery runtime and hardware protection are critical. Some participants experienced issues with their gloves. Of course, these interactions need consideration, which was not the case for our user study. A possible solution is to equip the bridge inspectors with special gloves for screen input or find other interaction techniques. We see no difference with the current status quo regarding the usage in bad weather, as pen and paper are also difficult to use under these conditions. From our perspective, every employed method offers advantages and disadvantages in that regard.
The interface design of our prototype is kept simple, so there is room for improvement. The buttons could benefit from additional text or tooltips. A guided tour or a tutorial would be beneficial to introduce new users to the program. Also, one can revise the design of the components. Currently, we used the Unity standard design and the standard colors. Not only the look of the application but also the interactions should be edited in the future. Should the application take a picture directly when pressing the damage recording button? The user benefits from a quick one-click solution, though the feedback showed that they had to get used to this. The way of inputting the damage is also a point of discussion: Some users prefer to input the damage via speech-to-text (if the environment is not too loud). Other experts wish to have a pencil with automatic handwriting recognition. Others liked the text-entry method as is for its clarity. From a technical perspective, the integration of autarkic solutions is possible due to the lack of network coverage inside the box girder.
We discussed the fusion of already existing software with the prototype several times during our user study with the participants. They see potential in more efficient processes – particularly for the debriefing and post-processing steps required. Maybe post-processing becomes even unnecessary in the future, as the documentation happens on-site and without media disruption. Potential with regard to a consistent, unique data collection solution could be tapped. Program interfaces and APIs are to consider also for enhancements of existing damage acquisition software (SIB-Bauwerke). These interfaces offer the possibility to exchange data between applications or provide new applications with data. We see three options for further developments: Interfaces, a further integration, or a harmonization to combine several data sources into the same data model. The contradiction of tradition and transcendence and their balancing as described by Ehn needs consideration at this point. New applications and workflows should respect existing traditions, while they will change the tradition by themselves, also reported as the “dialectical foundation of design” [13].
6.2 Model Quality
To display the model inside our prototype, we had to follow several preparation steps, including identifying the appropriate export and reduction options. There is a huge optimization potential for further automatization. To achieve that, exchange formats and workflows need further standardization and visualization environments like Unity should support them. This is also important with regard to openBIM, where file formats are specified that can be used by ideally every available tool.
In the case of the model quality, it was noticeable that some components in the BIM model were missing. Furthermore, it could be seen that the BIM model had the status of a planning model and was not available as an as-built model. This made it especially noticeable that five tendons were planned on both sides but only three per side were installed. These points were noticed by the participants early on and led to some questions.
We are confirmed in our first impression by the statements of the participants. An as-built model of the structure is required for testing the structure using such a demonstrator. Especially for details like floor openings, the number of tendons are important. This means that after completion, the structure is compared with the planning model in order to be able to hand over an actual twin of the real structure to the structure inspectors. However, this does not only refer to the geometric information but also to the component information, such as material and the other properties of the material. The more detailed the as-built model is developed, the more accurate the structural inspector can be in his assessment of a damage. There does not yet exist a model for every bridge, as we will discuss below. So there should be an option for these cases as well.
6.3 Localization
Regarding localization, the main discussion point was the precision of the tracking and the damage recording process. While marker tracking allows for high accuracy placement of virtual objects, SLAM offers only a limited accuracy due to the accumulation of errors. Thus, a combination of both seemed a viable option for us. However, one needs to install markers on the bridge. Vandalism, such as graffities, is a challenge in this regard. Another option would be to use labels or parts already present for tracking (e. g. position indicators for the different sectors of the bridge – if available). However, we expect a limited tracking accuracy due to the utilized iPad Pro. Experts disagree on the necessary precision or argue that this depends on the damage type and dimension. For example, one can rapidly identify a large-scale crack, even if the damage marker position is half a meter away from the actual damage. But if multiple defects spread in a dense distribution, in turn, experts desire a higher precision. In the last example, the size of the damage points is a problem too. The presented prototype gets unclear if multiple damage points are too close to each other, as they would overlap in that case. Clustering these damage points or a dynamic adoption of the sphere diameter would thus help to separate the damages. Taxonomies like Shneiderman’s information visualization seeking mantra could be helpful to revisit this idea [30].
The feedback also included the wish to move the damage point closer to the damage. In the current implementation, the damage position is at the location the user takes the picture. Our prototype could benefit from more flexible handling or separating the damage localization from the other processes. Although the latter would imply more interaction required by the user – providing them with more versatility.
6.4 Preparation and Debriefing
For the preparation and debriefing of a structural inspection, the demonstrator can be used in different ways. On the one hand, work can be done with the help of the iPad application and on the other hand with the desktop application based on AR or VR. The participants of the evaluation see a great added value here, especially in order to create a visual image of the damage in the environment beforehand, but also to review the damage again visually afterwards and to make a precise assessment. However, the participants still see a need for optimization in the presentation of the user interface and the input of texts. Furthermore, it is not possible to link notes to damages in the preparation phase, but these should not be included in the damage logging. In the post-processing, one participant sees the potential to carry out the damage assessment directly in the application to avoid duplicate maintenance between the application and SIB-Bauwerke, but the added value of the application is emphasized multiple times. However, participants also remarked that it takes more time to assess damages with our demonstrator, leading to a contradiction: On the one hand, participants see the potential to reduce post-processing effort, while the demonstrator sometimes also seems to be too time-consuming in-situ (at least for some of them). This reinforces the potential for other input methods and an optimized user interface and usability for future advancements.
Regarding the employed hardware, participants liked to have one device with all the information in one place. Nevertheless, additional tools, like hammers or crack measuring gauges will also be needed in the future. However, we can imagine a closer connection between digital tools such as our demonstrator and possibly digitized tools to directly gather data over protocols such as Bluetooth in the future.
Another important factor is the possible preparation of additional follow-up work, like creating tender specifications and plans. We see a lot of potential for further integration into organizational workflows here. In the understanding of BIM, it would be beneficial, if other parties involved in such processes could access the data with similar applications to preserve a central up-to-date BIM model and level of information.
6.5 Functionalities and Further Ideas
Several functionalities and ideas came up in our user study. First, the system’s awareness of the current damage status is requested. We could use colors, e. g. red, yellow, and green to color-code the state, like untreated, action required and solved. Another option would be the highlighting of damages. The data structure of our prototype can handle such information so that an extension would be feasible with little effort. Participants saw a danger in these “damage spheres”: One could get routine-blinded by only searching for the spheres instead of searching for existing damages. This would oppose to our main idea of supporting the bridge inspectors in their work. In our understanding, digital support should not lead to poor quality results or hinder the user.
With program interfaces or further integration into other programs, one can also achieve harmonization of texts. More functionalities like measuring on photos or zoom and filter of the camera image require additional data captured or increase the computational effort for the MR environment.
There does not exist a digital model for every structure and every bridge. Our prototype currently requires such a model and uses it for damage point localization and to align the virtual model with the real world. However, it would be possible to support even bridges without such a model. Thus, we could use a generalized, simple model as a rough orientation. In that case, the user could help by entering rough basic parameters such as the dimensions of the bridge and maybe a rough geometric estimation. The markers should be flexible in that case as well to allow the user a custom setup. Additionally, participants proposed to divide the model into smaller parts that can be partitioned between several bridge inspectors, especially for large bridges. That is closely related to the question, whether digital support makes sense for every structure size? While for larger bridges, it may be beneficial to have such a system, smaller bridges may not be worth the effort to setup an application like our demonstrator. For larger structures in turn, a even more detailed distribution could be advantageous.
We saw that we could transfer our prototype also to other use cases. One participant thought of possibly deploying such a system for the damage assessment at noise barriers. There, the damage types are less diversified than for bridges e. g. That would help us to build a more vertical prototype instead of a more horizontal one. Other potential application areas include tunnels, and buildings in general.
Also, participants posed the question on data security. The system can synchronize data whenever a connection is available. For our user study, we synchronized the data only when returning to the main menu. So, there is indeed an optimization potential concerning data safety when turning this demonstrator into a product. However, participants also reported examples where bridge inspectors lost their recordings due to bad weather conditions or paper loss.
7 Conclusion and Outlook
The evaluation of the MR prototype with real structural inspectors has shown that a combination of MR technology with the methodology of Building Information Modeling can provide support in the inspection of bridge structures. The potential for optimizing the entire inspection process, from preparation and execution to post-processing, by making relevant information available in bundled form and displaying and updating it in an understandable way became clear. Overall, the need for such digital support became particularly clear in order to enable the inspectors to focus more strongly on specialist experience and knowledge.
The quality of the 3D model has a significant part in the immersion of the bridge inspection application. In the evaluation an as-planned model of a bridge was used. This keeps some confusion among the bridge inspectors because the virtual models and the real bridge are not in the same state. In the future, an increase in model information towards the as-built model is recommended.
Currently, digitally supported bridge inspection by using AR and VR is an independent solution. This considers contents and tasks from DIN 1076 and should be more closely integrated here in the future. An approach would be the automated transfer of recorded damage into official inspection programs. Furthermore, an integration of AR methods directly into the official test programs to obtain a closed test routine. In addition, a connection with other databases should be considered. Here, important information for the inspection could be integrated into the AR test, such as monitoring data.
From the beginning of the developments, bridge inspectors were involved with their requirements for a support tool. This ensured that their needs were considered. At the same time, the acceptance of such a system was favored. This is essential, especially in the case of bridge inspection with its long tradition, to bring developments from research into practice in the future. It will also be crucial for further research to first demonstrate concepts and new processes on a pilot basis, as was done here, to enable tests and feedback from potential users. In this way and in the real environment, conclusions can be drawn as to whether a development is a real help for bridge inspection.
Regarding current and future research in this area, it will be particularly relevant to track the rapid development of digital technologies. The aim is to identify potential for optimized lifecycle management of bridge structures at an early stage and to harmonize it with best practices. Further support possibilities are seen, for example, in artificial intelligence for analyzing the data recorded during bridge inspections. This technology could be used to pre-select damage and provide information in an intelligent manner. There is also potential in terms of mapping and deriving the condition of the structure and its prognosis on the basis of large volumes of data. However, the bridge inspector, with their specialist experience and knowledge, always plays a central role in this. Thus, the combination of CSCW and MR (as already described by [14]) for BIM seems promising to us. In addition, the high safety standard of bridge inspection according to existing regulations must be maintained. Taking this into account, the digital support tools from research must ultimately also be incorporated into the regulations to find their way into practice. Here, especially the further adoption of the DIN 1076 is to mention in Germany. Similarly, purchasing processes for soft- and hardware in public domain need to reflect the usage of MR and VR. We argue that digital technologies support needs consideration in a user-centered way, regardless of the specific technology used.
The motivation to use digital support tools is high for the bridge inspectors, as our results show. Many of them are waiting for better and more suitable tools. One participant said: “We are working like pre-historic men, the system has great potential.” Others asked: “Where can we buy that?” For that reason, we gained the impression to have hit a nerve with the general idea of our demonstrator. We thus argue that there is great potential for further developments in this field. On the one hand the legal framework needs to be adapted accordingly, respecting the long-lasting tradition in the field of structural inspection. On the other hand, digital support tools like our presented demonstrator are needed as tools to make the task of bridge inspection easier for a human to perform [10] at the bridge or in the office, as bridge damages become an ever more pressing issue with increased traffic on our aging infrastructure.
About the authors
Urs Riedlinger is Research Associate in the group “Mixed and Augmented Reality Solutions” at Fraunhofer Institute for Applied Information Technology FIT in Sankt Augustin and PhD student at the “Promotionszentrum Angewandte Informatik” of the RheinMain University of Applied Sciences. He studied Electrical Engineering at Bonn-Rhein-Sieg University of Applied Sciences and completed his studies in 2017 with the Master of Engineering. His Bachelor and Master thesis covered Mixed Reality topics. Besides XR-projects he also worked on web-based collaboration research projects.
Florian Klein completed his Master studies in 2018 within the field of Automotive Engineering at the TH Köln – University of Applied Sciences. He then took over teaching duties in virtual product development and acquired new research proposals and projects as a Research associate at the CAD CAM centre of TH Köln. 2019 he became leader of the research and development projects and the field of VR applications at HHVISION. Besides the extraction of economic developments, he works on the integration of new methods and working methods in VR. Here, the core task is the examination between architecture and automotive engineering in Virtual Reality.
Marcos Hill is working for intecplan Essen (formerly LIST Digital GmbH & Co. KG) as BIM Manager since 2018. He works on the process consulting for digitalisation in the construction industry and the project management based on the BIM method. Furthermore, stocktaking, model-based documentation, and quality management based on the BIM method are his field of responsibility. He completed his studies 2018 at the Bochum University of Applied Sciences and became Bachelor of Science. Furthermore, he is speaker in lectures at Bochum University of Applied Sciences and University of Wuppertal.
Christian Lambracht is working for the city of Mönchengladbach as head of department for road construction and civil engineering since 2020. Before, he was working at the department for bridge construction and civil engineering at the civil engineering office of the city of Ratingen (since 2014) and for Schüßler-Plan Ingenieurgesellschaft in the department for bridge construction and civil engineering. He is member of the working group of the K-ING NRW (“Konstruktiver Ingenieurbau NRW”) since 2014 and self-employed for bridge inspections and structure restoration since 2019.
Sonja Nieborowski is Research Associate in the department of bridge and civil engineering at the German Federal Highway Research Institute. Working in the division “Maintenance of Engineering Structures she focuses on the topics of digital transformation in the life cycle of bridge structures, structural inspection, life cycle management and sustainability. Previously, she studied Mechanical Engineering at TH Köln – University of Applied Sciences and completed her studies in 2018 with a Master of Science.
Ralph Holst is working for the German Federal Highway Research Institute since 2002, after working for an engineering office (static, construction mangement) and for the Road Administration Schleswig-Holstein (2. state examination) and is responsible for the areas structural inspections and structure management. Since 2021, he is head of division “Maintenance of Engineering Structures”. He is active in national and international committees on these topics and has a lectureship at Bauhaus-Universität Weimar since 2019. For several years, topics like “innovative processes” and “digitalisation of structural inspections” play an important role for his work.
Sascha Bahlau is managing partner at the engineering office Intecplan Essen and has been working in project management and planning of construction and infrastructure projects for more than 10 years. After successfully completing his bachelor’s degree in civil engineering, Sascha Bahlau obtained a master’s degree in process and project management. He has already been able to contribute his extensive experience to publications on the subject of BIM on several occasions. Most recently in 2018, he published his collaboration with Klemt-Albert, K. “Evaluationen zu den Potenzialen von Building Information Modeling”.
Leif Oppermann leads the group “Mixed and Augmented Reality Solutions” in the Cooperation Systems department at Fraunhofer FIT in Sankt Augustin. Previously, he was Research Fellow and PhD student at the Mixed Reality Lab of University of Nottingham, UK, where he completed his doctoral studies in 2009 on location-based media. In 2003, he completed his studies on Media Informatics in Werningerode with a thesis on interactions in Augmented Reality on a Head Mounted Display with distinction. Currently, he is leading the 5x5G project “IndustrieStadtpark” on behalf of the Federal Ministry of Digital and Transport. His research interests include location-based applications, mobile Human-Computer Interaction, web-based collaboration, Mixed Reality, wearables and how to use all that in practice.
Acknowledgment
This paper is based on parts of the research project carried out at the request of the Federal Ministry for Digital and Transport, represented by the Federal Highway Research Institute, under research project No. 15.0666/2019/LRB including its final report „Structural inspections using 3D building models and augmented/virtual reality”. The authors are solely responsible for the content.
References
[1] Autodesk, 2003. Autodesk White Paper.Search in Google Scholar
[2] BCA Singapore, 2013. Singapore BIM Guide, Version 2 [WWW Document]. URL https://www.corenet.gov.sg/media/586132/Singapore-BIM-Guide_V2.pdf (accessed 5.28.20).Search in Google Scholar
[3] Bentley, R., Horstmann, T., Trevor, J., 1997. The World Wide Web as Enabling Technology for CSCW: The Case of BSCW. Comput. Support. Coop. Work CSCW 6, 111–134. https://doi.org/10.1023/A:1008631823217.10.1023/A:1008631823217Search in Google Scholar
[4] Bew, M., Richards, M., 2008. BIM maturity model. Presented at the Construct IT Autumn 2008 Members’ Meeting. Brighton, UK.Search in Google Scholar
[5] Bille, R., Smith, S.P., Maund, K., Brewer, G., 2014. Extending Building Information Models into Game Engines, in: Proceedings of the 2014 Conference on Interactive Entertainment, IE2014. Association for Computing Machinery, Newcastle, NSW, Australia, pp. 1–8. https://doi.org/10.1145/2677758.2677764.10.1145/2677758.2677764Search in Google Scholar
[6] Billinghurst, M., 2017. What is Mixed Reality? [WWW Document]. Medium. URL https://marknb00.medium.com/what-is-mixed-reality-60e5cc284330 (accessed 6.22.21).Search in Google Scholar
[7] Borrmann, A., König, M., Koch, C., Beetz, J. (Eds.), 2018. Building Information Modeling: Technology Foundations and Industry Practice. Springer International Publishing. https://doi.org/10.1007/978-3-319-92862-3.10.1007/978-3-319-92862-3Search in Google Scholar
[8] Broll, W., Lindt, I., Ohlenburg, J., Wittkämper, M., Yuan, C., Novotny, T., Fatah gen. Schieck, A., Mottram, C., Strothmann, A., 2004. ARTHUR: A Collaborative Augmented Environment for Architectural Design and Urban Planning. JVRB – J. Virtual Real. Broadcast. 1(2004). https://doi.org/10.20385/1860-2037/1.2004.1.Search in Google Scholar
[9] Brooks, Jr., F.P., 1987. Walkthrough—a Dynamic Graphics System for Simulating Virtual Buildings, in: Proceedings of the 1986 Workshop on Interactive 3D Graphics, I3D ’86. ACM, New York, NY, USA, pp. 9–21. https://doi.org/10.1145/319120.319122.10.1145/319120.319122Search in Google Scholar
[10] Brooks, Jr., F.P., 1996. The computer scientist as toolsmith II. Commun ACM 39, 61–68. https://doi.org/10.1145/227234.227243.10.1145/227234.227243Search in Google Scholar
[11] Chalhoub, J., Ayer, S.K., 2018. Using Mixed Reality for electrical construction design communication. Autom. Constr. 86, 1–10. https://doi.org/10.1016/j.autcon.2017.10.028.10.1016/j.autcon.2017.10.028Search in Google Scholar
[12] Cuong, N., Shim, C., Nguyen, Q., Jin, R., Jeon, C., 2020. Developing a mixed-reality based application for bridge inspection and maintenance.Search in Google Scholar
[13] Ehn, P., 1993. Scandinavian Design: On Participation and Skill, in: Schuler, D., Namioka, A. (Eds.), Participatory Design: Principles and Practices. Lawrence Erlbaum Associates, pp. 41–77.10.1201/9780203744338-4Search in Google Scholar
[14] Ens, B., Lanir, J., Tang, A., Bateman, S., Lee, G., Piumsomboon, T., Billinghurst, M., 2019. Revisiting collaboration through mixed reality: The evolution of groupware. Int. J. Hum.-Comput. Stud., 50 years of the International Journal of Human-Computer Studies. Reflections on the past, present and future of human-centred technologies 131, 81–98. https://doi.org/10.1016/j.ijhcs.2019.05.011.10.1016/j.ijhcs.2019.05.011Search in Google Scholar
[15] Federal Ministry of Transport and Digital Infrastructure, 2015. Road Map for Digital Design and Construction [WWW Document]. URL https://www.bmvi.de/SharedDocs/DE/Publikationen/DG/stufenplan-digitales-bauen.pdf?__blob=publicationFile (accessed 2.24.17).Search in Google Scholar
[16] Fonseca, D., Redondo, E., 2013. Are the architecture students prepared for the use of mobile technology in the classroom?, in: Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality, TEEM ’13. Association for Computing Machinery, New York, NY, USA, pp. 481–487. https://doi.org/10.1145/2536536.2536610.10.1145/2536536.2536610Search in Google Scholar
[17] Garbett, J., Hartley, T., Heesom, D., 2021. A multi-user collaborative BIM-AR system to support design and construction. Autom. Constr. 122, 103487. https://doi.org/10.1016/j.autcon.2020.103487.10.1016/j.autcon.2020.103487Search in Google Scholar
[18] Hansen, L.H., Kjems, E., 2019. Augmented Reality for Infrastructure Information: The eCAADe and SIGraDi Conference 11–13 sep 2019, Faculty of Architecture, University of Porto, Portugal. Archit. Age 4th Ind. Revolut. 2, 473–482.Search in Google Scholar
[19] Harrison, S., Dourish, P., 1996. Re-place-ing space: the roles of place and space in collaborative systems, in: Proceedings of the 1996 ACM Conference on Computer Supported Cooperative Work, CSCW ’96. Association for Computing Machinery, New York, NY, USA, pp. 67–76. https://doi.org/10.1145/240080.240193.10.1145/240080.240193Search in Google Scholar
[20] HM Government, 2012. Building Information Modelling [WWW Document]. URL https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/34710/12-1327-building-information-modelling.pdf (accessed 2.6.17).Search in Google Scholar
[21] Messner, J.I., Yerrapathruni, S.C.M., Baratta, A.J., Whisker, V.E., 2003. Using Virtual Reality to Improve Construction Engineering Education, in: The Pennsylvania State University (Ed.), Presented at the American Society for Engineering Education Annual Conference & Exposition.Search in Google Scholar
[22] Milgram, P., Takemura, H., Utsumi, A., Kishino, F., 1995. Augmented reality: a class of displays on the reality-virtuality continuum, in: Telemanipulator and Telepresence Technologies. Presented at the Telemanipulator and Telepresence Technologies, International Society for Optics and Photonics, pp. 282–292. https://doi.org/10.1117/12.197321.10.1117/12.197321Search in Google Scholar
[23] Moon, H., Kim, Hyeon-seong, Moon, S., Kim, Hyeun-wook, Kang, L., 2012. Prototype model of wireless maintenance operation system for bridge facility using VR and ubiquitous technology, in: Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI ’12. Association for Computing Machinery, New York, NY, USA, p. 350. https://doi.org/10.1145/2407516.2407601.10.1145/2407516.2407601Search in Google Scholar
[24] Oppermann, L., Shekow, M., Bicer, D., 2016. Mobile Cross-media Visualisations Made from Building Information Modelling Data, in: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, MobileHCI ’16. ACM, New York, NY, USA, pp. 823–830. https://doi.org/10.1145/2957265.2961852.10.1145/2957265.2961852Search in Google Scholar
[25] Piekarski, W., Thomas, B.H., 2001. Tinmith-Metro: new outdoor techniques for creating city models with an augmented reality wearable computer, in: Fifth International Symposium on Wearable Computers, 2001. Proceedings. Presented at the Fifth International Symposium on Wearable Computers, 2001. Proceedings, pp. 31–38. https://doi.org/10.1109/ISWC.2001.962093.10.1109/ISWC.2001.962093Search in Google Scholar
[26] Prouzeau, A., Wang, Y., Ens, B., Willett, W., Dwyer, T., 2020. Corsican Twin: Authoring In Situ Augmented Reality Visualisations in Virtual Reality, in: Proceedings of the International Conference on Advanced Visual Interfaces, AVI ’20. Association for Computing Machinery, New York, NY, USA, pp. 1–9. https://doi.org/10.1145/3399715.3399743.10.1145/3399715.3399743Search in Google Scholar
[27] Riedlinger, U., Oppermann, L., Prinz, W., 2019. Tango vs. HoloLens: A Comparison of Collaborative Indoor AR Visualisations Using Hand-Held and Hands-Free Devices. Multimodal Technol. Interact. 3, 23. https://doi.org/10.3390/mti3020023.10.3390/mti3020023Search in Google Scholar
[28] Sanchez-Sepulveda, M.V., Marti-Audi, N., Fonseca-Escudero, D., 2019. Visual Technologies for Urban Design Competences in Architecture Education, in: Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality, TEEM’19. Association for Computing Machinery, León, Spain, pp. 726–731. https://doi.org/10.1145/3362789.3362822.10.1145/3362789.3362822Search in Google Scholar
[29] Schall, G., Junghanns, S., Schmalstieg, D., 2010. Vidente-3D Visualization of Underground Infrastructure using Handheld Augmented Reality. Geohydroinformatics Integrating GIS Water Eng.Search in Google Scholar
[30] Shneiderman, B., 1996. The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations, in: Proceedings of the 1996 IEEE Symposium on Visual Languages, VL ’96. IEEE Computer Society, Washington, DC, USA, p. 336.Search in Google Scholar
[31] Sutherland, I.E., 1964. Sketchpad a Man-Machine Graphical Communication System. SIMULATION 2, R-3. https://doi.org/10.1177/003754976400200514.10.1177/003754976400200514Search in Google Scholar
[32] Vajna, S., Weber, C., Schlingensiepen, J., Schlottmann, D., 2013. CAD/CAM für Ingenieure: Hardware, Software, Strategien. Springer-Verlag.Search in Google Scholar
[33] van Nederveen, G.A., Tolman, F.P., 1992. Modelling multiple views on buildings. Autom. Constr. 1, 215–224. https://doi.org/10.1016/0926-5805(92)90014-B.10.1016/0926-5805(92)90014-BSearch in Google Scholar
[34] Wang, X., Schnabel, M.A. (Eds.), 2009. Mixed Reality In Architecture, Design, And Construction.10.1007/978-1-4020-9088-2Search in Google Scholar
[35] Woodward, C., Hakkarainen, M., 2011. Mobile Mixed Reality System for Architectural and Construction Site Visualization, in: Ching Nee, A.Y. (Ed.), Augmented Reality. InTech, pp. 115–130. https://doi.org/10.5772/26117.10.5772/26117Search in Google Scholar
[36] Wu, Z., Wang, L., Fu, Z., Zhu, L., Dou, F., Xu, P., 2018. VR+BIM: Perception and Design Optimization of Highway, in: 2018 International Conference on Virtual Reality and Visualization (ICVRV). Presented at the 2018 International Conference on Virtual Reality and Visualization (ICVRV), pp. 164–165. https://doi.org/10.1109/ICVRV.2018.00059.10.1109/ICVRV.2018.00059Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston
