Papers
Developing An Augmented Reality System With Real-Time Reflection Using Real-Time Ray Tracing Technique For Landscape Design Visualization
Hao Chen Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University
Tomohiro Fukuda Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University
Nobuyoshi Yabuki Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University
In the landscape design process, it is crucial to review the context between design targets and their surrounding environment, if inadequate, the loss of time and social resources will be caused in the subsequent construction process. Augmented reality (AR) technology can visualize the relationship between a design object and its surrounding environment (Giunta et al. 2018). One of the current problems in applying AR to the design process is the lack of inherent interaction between the design model and its surrounding environment. One of them is the rendering of physical phenomena that a design model exerts on its surrounding environment, and although the phenomenon of reflection and refraction on water surface and glass is one of the essential design elements in landscape design (Booth 1989), it cannot be achieved with existing AR systems. Although basic reflective representations with 3D models mirrored on reflective surfaces have been realized by AR systems ([Authors 2021]), they cannot support reflection phenomena occurring on multiple planes, and the accuracy of reflections is not satisfactory. To overcome this limitation, the objective of this study is to develop an AR system for landscape design review that enables more precise reflection between the 3D virtual model to be designed and the physical environment on multiple reflective surfaces. The hardware configuration in this research is a webcam and a PC, and the implementation of reflection in AR uses the real-time ray tracing rendering technology (Fredriksson and Scott 2016), which can create highly realistic reflections compared to traditional rasterization rendering (Lagarde and Zanuttini 2013). The proposed system imports an AR software development kit (SDK) into the game engine to implement the basic AR effect. An AR content developer manually defined the reflective plane in the game engine to calculate the reflection image on it, based on the position of the reflective plane in the physical environment, and then the real-time ray tracing function can be executed in the game engine. To ensure that the reflective content blends into the physical environment, we customized the shaders used for real-time ray-tracing rendering so that when rendering a reflective surface, all pixels except for those in the reflected image are transparent. As a result, the AR output result is that the predefined planar model blends into the physical environment and only the reflected image remains. Finally, the results obtained in this study are compared with the ground truth images and the results obtained in previous studies ([Authors 2021]). The method proposed in this study using real-time ray tracing can achieve more accurate reflection effects for multiple planes between the virtual model and the physical environment in AR. Future work includes automating the process of generating reflective surfaces to make the whole process more efficient. Giunta, L., O’Hare, J., Gopsill, J., and Dekoninck, E. 2018, ‘A review of augmented reality research for design practice: looking to the future, DS 91: Proceedings of NordDesign 2018, Linköping, Sweden, 14th-17th August. Booth, N. 1990, Basic elements of landscape architectural design: Waveland press, p.257, Long Grove, Illinois, USA. Fredriksson, J., and Scott, A. 2016, ‘Generating Real-time Reflections by Ray Tracing Approximate Geometry’, Master’s thesis, Department of Applied Information Technology, Chalmers University of Technology, Göteborg, Sweden. Lagarde, S. and Zanuttini, A. 2013, Practical Planar Reflections Using Cubemaps and Image Proxies. GPU Pro: CRC Press, p.51-68, Boca Raton, Florida, USA. [Authors 2021]: This is omitted due to double-blind peer review and will be clearly stated at the camera-ready submission stage.
Keywords: Augmented Reality (Ar), Reflection, Landscape Design, Interactive Visualization, Real-Time Rendering, Planar Reflection, Real-Time Ray Tracing, Sdg11: Sustainable Cities And Communities.