Technology for the Winter Olympics|Takamatsu Ling-BiO-MR Online Viewing Interactive System

Project Background & Introduction

In the key special project of "Science and Technology for Winter Olympics" of National Key R&D Program, Mr. Fu Zhiyong's team researches the 3D construction and semantic understanding technology of large-scale competition scenes based on multi-source data, and builds a mixed reality e-sandbox and data visualization engine to support the situation of the Winter Olympic Games. Research and build a mixed reality electronic sandbox and data visualization engine to support the visualization of information. To help the main operation center of the Winter Olympic Organizing Committee, the regional command and the command center of each competition venue to fully understand the scene situation, to assist the relevant personnel in the complex environment of cross-regional, cross-level communication and cooperation, for the Winter Olympic Games command system in the event preparation, event operations, opening and closing ceremonies and other scenarios to smoothly carry out security monitoring, logistical support, emergency response to emergencies and other work to provide support.

In February 2022, the team received an official letter of appreciation from the Beijing Winter Olympic Organizing Committee. Since then, the team members have been working on their own designs based on their research results and project experience, and we would like to take this opportunity to introduce the team members and their design works in a series of topics on the Winter Olympics.

Author Bio

Songling Gao

 Master students

Tsinghua University, School of Fine Arts, Class of 2019, Master's Degree in progress

Bachelor, College of Design and Art, Beijing Institute of Technology

He graduated from Beijing Institute of Technology, College of Design and Art, majoring in product design. In 2019, he was recommended to enter Tsinghua University, College of Fine Arts, majoring in information art and design, supervised by Associate Professor Fu Zhiyong. In 2022, he created his graduation design "BiO-MR online interactive system".

Interview Content

Q1

When did you get involved with the Winter Olympics?

In my first semester of graduate school, I participated in a project on technology for the Winter Olympics. The project focused on mixed reality-based technologies for visualizing and conferring on the Winter Olympics situation. During the research process, we designed and produced a digital information screen for Winter Olympics situational awareness, and at the same time, we designed and produced a Winter Olympics sandbox for alpine skiing based on mixed reality environment, etc. Through these project practices, I got a deeper understanding of mixed reality environment, and I used this topic as the background of my final project for the extension and application of practical scenarios.

Figure 1 Technology Winter Olympics results

Q2

Why did you choose to do this project?

I hope to provide a way of thinking for the digital transformation of sports events online through the expression of mixed reality technology, to stimulate public interest and enthusiasm in ice and snow culture with interactive and fun interactive experiences, to promote the goal of "300 million people participating in ice and snow sports", to accelerate the promotion and popularization of ice and snow sports in the context of the 2022 Beijing Winter Olympics, and to convey the ice and snow culture to people with the power of design. It will accelerate the promotion and popularization of ice and snow sports in the context of the 2022 Beijing Winter Olympics, help the rapid development of China's ice and snow industry, and deliver ice and snow culture to more people with the power of design.

Q3

What is the main focus of your work?

On the choice of scenario, I mainly focus on the scenario of online viewing. On the one hand, due to the epidemic, many spectators are unable to watch the events on site, and digital service means provide more possibilities for online viewing; on the other hand, as China is the host country of the Winter Olympics in 2022, the ice and snow culture is becoming popular and more and more people pay attention to it. Therefore, I hope to provide a better experience for spectators to watch sporting events online through my research project, and also to extend the research results of technology Winter Olympics to practical application scenarios.

Q4

The research significance of your work?

In the theoretical research, contextual cognition and mind flow theory are used as guidance, adding the user's psychology and the environment they are in to the research, integrating emotional experience with the event content, rising from focusing on the functional level of design to the emotional experience level, positioning the design direction of the system in the contextual experience, delivering the event content to the user better, expanding the perception channels, making the user's pleasure and satisfaction in the environment The system is designed to improve the user's sense of pleasure and satisfaction in this environment, and to expand the possibility of online game viewing.

Q5: The

How was the research conducted in the early stages?

I have conducted preliminary research mainly from three aspects: theory, technology, and users.

In terms of theory, we mainly studied contextual cognitive theory and mental flow theory. Contextual cognitive theory was proposed as early as pragmatism until the 1990s when it was first formed. Contextual cognitive theory says that knowledge is contextualized, just like we cannot learn to recognize words by dictionaries alone, we have to learn in actual scenes. There is a constitutive relationship between knowledge, context, and behavior. This theory was proposed by Checkson Micha in 1975, which means that when users are fully engaged in an activity, their attention is highly concentrated and not influenced by irrelevant factors, and the Flow state is generated during the experience, this Flow is not an emotional high point all the time, but a dynamic change in the curve, emphasizing the sense of rhythm in the process, and constantly hitting the user's experiential contacts through different behaviors This Flow is not a constant emotional high point, but a dynamic change in the curve, emphasizing the sense of rhythm in the process, and constantly hitting the user's experiential touch points through different behaviors, so that participants are continuously immersed in the activity. The process of watching the game itself is also rhythmic, so it is necessary to provide the right information at the right time.

Figure 2 Technology Winter Olympics results

In terms of technology, combined with the research of the preliminary Winter Olympics project, MR technology is different from AR technology, which simply superimposes virtual and real information, while MR technology environment allows users to interact and perceive in the new environment generated by the fusion of virtual and real. A typical device is the HoloLens glasses, a mixed reality device, which users wear to complete information acquisition through basic gestures. Information is presented in a wide range and can be arranged in many different ways. For the needs of information overlay like in the process of watching a game, it is possible to take advantage of this spatial environment to present and obtain information.

Figure 3 Mixed Reality Technology

In terms of user research, I researched professional fans and ordinary viewers who do not know the game, and drew a user portrait based on their characteristics. Through the research, I understood which content is the moment they are most interested and most fully engaged, and which moments and which content they are easily distracted and bored. I found that different scenes of the whole game watching process have different emotional and state reactions for users. As a result, I divided a game into five main scenes, namely scrimmage, goal, penalty, overtime, and replay scenes. Since different scenes correspond to different contents, the behavior of users is different.

Figure 4 Theoretical research structure

With the preliminary research on theory, technology and users, I built the architecture of my initial research, firstly based on contextual cognition, there is a constitutive relationship between knowledge, context and behavior, then with reference to the atomic theory of flat information organization structure, I explored the organization form of spatial information, from cellular organization, model scenes, to form the final space. Corresponding with the contextual cognitive theory, it is found that knowledge corresponds to the information content of the interface, and the interface is composed of the smallest cells, and different cells form different arrangements of organization to be presented in the spatial environment. Then, the context corresponds to different model contents and different scenes. For example, the scrimmage scene, which runs through the whole process of the game and lasts for a long time, has less key information and more scattered information, so the users will get tired in this context. The solution to the behavioral layer is to increase user interactions and auxiliary information to compensate for the lack of focus on the scene content, which leads to user boredom and burnout, and to improve the user's reliance and immersion in the system.

From this, it can be seen that different scene contents will produce different scene characteristics, and different scene characteristics correspond to different user behavior interactions. Through different rhythm changes, information is presented, which can make the audience have a good viewing experience. Above, are some of the work I did in the preliminary research. Through these contents, I have a basic theoretical structure, which provides a strong theoretical support for the subsequent design practice.

Q6: The

What is the specific design process?

My prototype system is designed to allow "amateur viewers" who do not know the game to see the game through this system. With the help of mixed reality devices, I designed a prototype system for watching hockey games online, mainly consisting of two parts: watching the game and learning to improve, each with corresponding functional content.

Figure 5 System content architecture

On top of the information presentation in space, with the advantage of spatial environment to express information, the information is divided into three layers. The top navigation layer is the entrance of each functional module, the user completes the module selection through the action of raising the head, and the icon in the top navigation area will give the user feedback through different states. The middle information panel layer is the information panel corresponding to each functional module, which is divided into two kinds of browsing information and interactive information. The bottom model layer is the information content related to the stadium model, including the venue, route, player distribution, etc. Each of the three layers has its own role to complete the presentation of different information.

Figure 6 System information hierarchy

Q7: The

Please share your practical approach in your projects.

The system development of this design is based on Unity 2019.1.7f1 version, the programming language is based on C#, and the programming IDE (Integrated Development Environment) is based on Visual Studio 2019. HoloLens 2, Microsoft open-sourced the Microsoft Mixed Reality Toolkit (MRTK) mixed reality program package. During the development of the system, this design primarily utilizes the MRTK base dependency package Microsoft.Mixed Reality.Toolkit.Unity.Foundation.2.7.3; Microsoft.MixedReality.Toolkit.2 with tools that can be used to create mixed reality experiences. Tools.2.7.3 toolkit; the Microsoft.MixedReality.Toolkit.Unity.Extensions.2.7.3 extension package with add-ons to extend MRTK functionality; and the design uses DOTween Pro v1.0. 175 animation plug-in, DOtween is a fast and efficient object-oriented animation engine, which can help achieve a variety of animation effects in Unity engine easily and quickly. In addition, the software platform used for graphic design, this design is based on Adobe Illustrator 2019, Adobe Photoshop 2019 and Figma for graphic design.

Figure 7 Development technology platform

Q8

How do you feel about mixed reality devices through your design practice?

It was a new attempt and idea to realize such a demand for online game watching with the help of mixed reality technology environment. After the design was completed, I conducted a user test and found that most of the users had a strong sense of novelty and desire to experience the game through this device, and most of the participants were curious about this device. But through the user testing and feedback, I also found some problems, first, although people are willing to try, but for novice users, it is difficult to start operating the system, even if I added a novice guidance teaching, but the space gesture operation is less accurate, users sometimes have a sense of frustration, and do not want to try again. Secondly, the device itself has some limitations, the operation time is too long the machine will be very hot, and every time the user needs to change a device angle calibration and other issues, these problems in the future are continuously optimized.

Figure 8 User testing

Q9

What problems did you encounter in the research process and how did you solve them?

I spent some time on learning the theory in the early stage, I hope this design practice of mine is not only functional implementation, but some theory to guide the subsequent design output, therefore, in the early stage, I kept looking up literature and books, etc., discussed with my mentor many times, sorted out my ideas, and finally formed a basic structure that can guide my subsequent design, I think this step is very important and critical. Only if your research ideas are clear, the subsequent output will be relatively smooth, so I would like to say, do not start sketching the design at the beginning, but to clarify the ideas, with the help of what kind of theory, what problems are solved, which people are targeted, what is the basic structure, etc. These are clear, the design practice is constantly optimizing the effect, and finally reach a more satisfactory state.

Results show

Here's a brief video of what my entire prototype system consists of!

Practice Reflection

Q11: The

One of the takeaways from the entire design practice?

From initial theoretical research to research to design practice, I have gained a deeper understanding of information presentation in mixed reality environments. I have also gained a lot of insights in the design process, which I have summarized and refined into 6 design guidelines, and I hope to bring a little reference for future students studying related fields.

The main two parts are layout and interaction. In the layout organization: firstly, I organize information based on contextualized vein, and establish the system architecture of spatial information from cell formation organization then to model scene and finally to space; secondly, in the information presentation, multi-layer three-dimensional presentation is adopted, and each of the three layers of information has its own function; thirdly, the current focus information is guided through points of interest, and although the spatial information is presented in a wider range, attention should be paid to the main and secondary, to reduce interference of focused information.

Figure 9 Design Guidelines

In terms of interaction logic, firstly, a panoramic information hierarchy is established to minimize excessive nesting of layers. The mixed reality environment is not as accurate as flat information clicking, so the interaction language can increase Hover and reduce Click state, so that the whole operation will be more smooth. Second, I took advantage of the spatial environment information presentation and designed a star dome style navigation to complete the selection of functional modules through the action of raising the head. Thirdly, since most users are unfamiliar with this hybrid environment, newbie guidance is very necessary. I help users adapt to the system by teaching guidance with a sense of scene, and good teaching guidance determines the first impression of whether users can continue to use this system.

Q12: The

What are your future prospects for this research project?

At present, this prototype system is only limited to the hockey game, and we hope to introduce more online game scenarios in the future to expand the possibilities of online game watching. By summarizing the commonality and individuality of online games, we will develop a targeted online game viewing interactive system suitable for different games and greatly enhance the online game viewing options for participants.

Building spatial information design model based on contextual cognitive theory plays a great role in promoting the solution to the problem of limited display of two-dimensional flat spatial information, which is an attempt experienced in the process of continuous development of digitalization, which involves the joint research of multiple disciplines such as design, psychology and computer engineering, and has certain research significance for the future expansion of information presentation, which needs to be explored together!

Team Members

Team Leader.

Fu Zhiyong Guan Yan

Team members.

Tao Shu, Gao Songling, Xu Jingwen, Zhang Jin, Lei Jieming, Song Zhengyin, Wang Qirui, Wu Hongfei

Scroll to Top