Technology for Winter Olympics|Jingwen Xu - Research on the design of virtual reality interactive experience of Olympic heritage

Project Background & Introduction

In the key special project of "Science and Technology for Winter Olympics" of National Key R&D Program, Mr. Fu Zhiyong's team researches the 3D construction and semantic understanding technology of large-scale competition scenes based on multi-source data, and builds a mixed reality e-sandbox and data visualization engine to support the situation of the Winter Olympic Games. Research and build a mixed reality electronic sandbox and data visualization engine to support the visualization of information. To help the main operation center of the Winter Olympic Organizing Committee, the regional command and the command center of each competition venue to fully understand the scene situation, to assist the relevant personnel in the complex environment of cross-regional, cross-level communication and cooperation, for the Winter Olympic Games command system in the event preparation, event operations, opening and closing ceremonies and other scenarios to smoothly carry out security monitoring, logistical support, emergency response to emergencies and other work to provide support.
In February 2022, the team received an official letter of appreciation from the Beijing Winter Olympic Organizing Committee. Since then, the team members have been working on their own designs based on their research results and project experience, and we would like to take this opportunity to introduce the team members and their design works in a series of topics on the Winter Olympics.

Author Bio

Jingwen Xu
Master students

Master of Interdisciplinary Studies, Department of Information Art and Design, Academy of Fine Arts, Tsinghua University, Class of 2019
Bachelor of Business Administration, School of Economics and Management, China Agricultural University, Class of 2015

From Taizhou, Jiangsu Province, she enrolled in the Department of Information Art and Design Interdisciplinary at Tsinghua University School of Fine Arts in 2019, under the supervision of Associate Professor Fu Zhiyong and co-supervisor Associate Professor Yu Chun. During her master's degree, she focuses on the cross-fertilization of art design and new media, exploring the possibility of information interaction and multi-sensory experience. She mainly participated in AI CITY and Technology Winter Olympics projects and published papers, and her project result "A-TOPIA" participated in the global youth creation program and won awards, and was also exhibited in the 2020 Design Future International Exhibition. 2022, she created her graduation work "The Realm of Remembering Olympics" based on the Technology Winter Olympics project.

Interview Content

Q1: When did you get involved with the Technology Winter Olympics?

I have been involved in the project team of the Winter Olympics since 2020, focusing on the sub-project "Mixed Reality-based Winter Olympics Situational Visualization and Consultation Technology", during which I have conducted a series of researches and realized the application transformation, including the preparation of the "Winter Olympics Situational Visualization Interaction and Visual Design Specification", the design and development of the Winter Olympics information screen and During this period, we have conducted a series of research and application transformation, including the preparation of the "Winter Olympic Games Scene Visualization Interaction and Visual Design Specification", the design and development of the Winter Olympic Information Screen and the Genting Snow MR Sandbox, and writing a paper based on the research. Based on the large amount of Olympic-related information materials and related technologies brought by the technology Winter Olympics project, I also found more extendable possibilities and applied them to the subsequent graduation design creation to achieve a more free way to present the combination of content, interaction and experience.

Q2 What are the main issues that your work focuses on?

The digitization of heritage requires consideration not only of the content and form of heritage, the choice of information carrier, the presentation of visual narrative and its relationship with the audience, but also the communication of information, the understanding and reflection of the audience. How to build a heritage experience space for Olympic heritage, evaluate the meaningful expression and perceptual experience through metaphorical design and transformation of interaction paradigm with the help of VR technology, so as to reach the goal of Olympic culture communication and establish the connection of Olympic spirit, is the main problem to be solved in this work.

Q3 The research significance of your work?

On the one hand, we hope to promote the heritage of Olympic heritage out of museums and into daily life, reconstruct and interpret the content of heritage by means of easy and pleasant gamification, and explore the positive effects of Olympic heritage. On the other hand, it gives a virtual space experience design method and paradigm for Olympic heritage, and expands the possibility of digital experience of Olympic heritage, especially for intangible heritage, and can be applied to more types of cultural heritage content.

How did you conduct your research in the early stages?

My research revolves around Olympic heritage and multi-sensory experience in virtual space. The Olympic heritage research includes the definition and classification of Olympic heritage, the research and analysis of the handling cases of heritage and the content of each type of heritage, and the formation of Olympic heritage memory signal resource/library. Based on the theory of multi-sensory experience and symbolic metaphor, the construction of virtual space, the perception of virtual space and multi-sensory experience are studied, and the application of virtual space metaphor design methods and associative experience are explored, and then a research model is formed, which lays the theoretical foundation of this work on the virtual interactive experience of heritage.

User research was conducted in the early stage of the design, questionnaire research and workshops were carried out to extensively collect people's knowledge about Olympic heritage and what heritage content people are concerned about. The workshop revolved around my Olympic memories, through brainstorming and using design tools to build the Olympic heritage space and access the Olympic heritage resource base as the basis for the design.

What is the specific design process?

At the early stage of design, five major heritage contents were extracted from the Olympic heritage resource base as the themes of five heritage spaces, and then subdivided into a number of images in turn to form the content organization level and information hierarchy of the heritage to be presented in the Olympic memory.

After several iterations of sketching, the spatial design of "Yi'ao Realm" was finalized and further developed into a 3D model. The entire space is composed of five sub-scenes, connected by a central square. Each thematic scene has a different structure according to its metaphorical connotation and experiential tendency. The design elements in the space are derived from the visual transformation of the Olympic culture/memory/resource base, and a number of symbols and information contents are extracted to form the components in the space, and the connotation is explained to the experiencer through the feedback in the interactive process.

The experience flow of the "Memory of Olympics" space is shown in the figure below. The space of Olympic image opens the experience of memory exploration, capturing the Olympic memory fragments of people with different identities in the spiral weightless space, piecing together and restoring the colorful Olympic stories, and witnessing all the Olympic images; the space of Olympic sound opens the experience of visual concert, enjoying the Olympic songs and perceiving the rhythm, atmosphere and emotion of the Olympics in the process of clicking, floating and combining geometric shapes; the space of Olympic pavilion opens the experience of switching scenes in the blind box, bringing freshness and stimulating the desire for exploration between the mapping and switching of the earth and the twisted egg, and perceiving the changes of the previous Olympics and understanding the Olympic Games. The space of Olympic Road opens the experience of celestial movement, based on the control of each "celestial" element by handle operation, restores its real movement state, and conveys the power and beauty of each Olympic project.

The experience cycle of the "Olympic memory" space is gradual, from the overall experience to the medium scene experience to the local experience, which is the process of establishing the relationship between the experience and the "Olympic memory". In the cycle of the overall experience, the experience user can see five scene modules from the viewpoint of the central square, forming a three-dimensional Olympic memory bank; when the user roams to a sub-scene, he/she enters the experience of the medium scene, and the levels of experience are: symbol, metaphor, experience, perception and re-exploration; when the user interacts with the elements in the scene, he/she enters the local experience, and the experience user goes through the process of connection, orientation, interaction, touching and transmission in turn. When the user interacts with each element in the scene, he or she enters into the local experience.

What paradigm of virtual space experience design does your research give and how does it integrate with creative practice?

In the process of design practice, I extracted five design patterns: jungle exploration pattern, memory fragment pattern, shape combination pattern, random egg twisting pattern and celestial movement pattern, and applied them in each of the five sub-scenes of "Olympic memory" to provide a usable experience for the Olympic heritage multi-sensory space for youth. The solution includes both the problem to be solved and the desired effect to be obtained, and is applicable to different types of Olympic heritage content.

Q7 How did the prototype of your work come to fruition?

The game engine Unity is used to build the prototype of "Yi'o Realm". After the scene model and dynamic effects are completed in Blender, they are placed in Unity for integration and adjustment, building the environment, writing interaction events, and combining with Oculus' developer mode to assist virtual reality development and realize virtual reality experience.

Q8 Why choose virtual reality technology to assist in research and creation?

Today's world has entered the era of information technology, and with the rapid development of information technology, many new technologies including MR (Mixed Reality), VR (Virtual Reality), AR (Augmented Reality) have been generated. VR differs from MR and AR in its virtual nature, which is a three-dimensional three-dimensional environment generated by the use of computers, a virtual world isolated from reality, which can isolate interference and provide a strong spatial immersion. From the perspective of cross-fertilization of technology and art, the creation and development of VR technology provides a new path for the creation of art beyond the real world in space and time, expanding the dimension of sensory integration and information interaction, and can bring more possibilities for heritage transmission and preservation while creating virtual worlds.

Results show

My graduation work--"Memory of the Olympics" virtual reality interactive experience space, transforming the Olympic heritage into a multi-level information structure, building different interactive experience levels, combining five design modes to form a layered progressive experience cycle, building an immersive interactive space with the help of virtual reality technology, guiding experiencers to explore different metaphors, interpreting the application of multi-sensory space design methods, and evaluating the meaning expression and perception experience of the heritage. The immersive interactive space is constructed with the help of virtual reality technology, guiding the experience participants to explore different metaphorical connotations, interpreting the application of multi-sensory space design methods, and evaluating the meaningful expression and perceptual experience of the heritage.

Research Summary

This study summarizes the design method of virtual space multi-sensory experience applicable to heritage conservation based on heritage research, multi-sensory experience research with symbolic metaphor theory and virtual reality technology achievements, and proposes a research model of multi-sensory space for Olympic heritage content. On the basis of this, a model and prototype paradigm of multi-sensory space for Olympic heritage conservation is constructed by combining the creation practice, which provides a new way of thinking for the interpretation and conservation of intangible heritage, explores a natural multi-level interactive experience to mobilize multiple perceptions of people, and provides a reference method case for the inheritance and conservation of more kinds of heritage.

Future Outlook

In further research, there are still directions that can be expanded. On the one hand, there are more possibilities for the use and integration of multiple senses in virtual space. In the current scenario, the body senses used are mainly visual, auditory and tactile, while smell and taste also play a great role in the interaction between people and the environment; the emotions and psychology expressed are more positive, while the heritage does not only have positive content, but also negative or negative content, which is also an integral part of the heritage. On the other hand, this interactive experience model can be used for other types of heritage, such as Olympic policy, economic, social and environmental heritage, traditional arts heritage, singing and dancing heritage, agricultural heritage, etc., so as to create richer examples of heritage digitalization, enrich the content of heritage experience, further verify the usefulness of heritage interactive experience model in the process of virtual space experience design, and expand more interactive experience possibilities.

Team Members

Team Leader.
Fu Zhiyong Guan Yan

Team members.
Tao Shu, Gao Songling, Xu Jingwen, Zhang Jin, Lei Jieming, Song Zhengyin, Wang Qirui, Wu Hongfei

Scroll to Top