ETRI Develops Immersive Extended Reality Integrating Virtual and Augmented Reality

The Electronics and Telecommunications Research Institute (ETRI) announced on the 19th that it has developed an 'immersive extended reality (XR)' technology that combines virtual reality (VR) and augmented reality (AR).

The core of immersive XR is the three-dimensional precise motion analysis of multiple users and user position estimation technology in complex spaces. The multi-user 3D precise motion analysis is performed by inferring the user's 3D joint position information based on 2D video data using deep learning technology in a mobile environment.

Conceptual diagram of 'Immersive Extended Reality (XR)' technology. Provided by Electronics and Telecommunications Research Institute (ETRI)

Previously, this required system resources equipped with expensive graphics processing units (GPUs), but ETRI has enabled efficient extraction of 3D posture information using the user's smartphone.

Additionally, user position estimation in complex spaces is a technology to naturally integrate augmented reality content into the real world. This technology extracts features from video using a smartphone camera, then generates and calibrates a 3D point cloud to build an environmental map, allowing precise estimation of the user's device location.

The immersive XR is designed to maximize user immersion in AR environments. According to ETRI, this allows precise interpretation of the user's environment and actions in VR and AR environments, enabling multiple users to experience interactions of the same quality without separate restrictions.

ETRI also created pilot content that allows multiple users to participate in XR experiences locally and remotely by creating virtual reality spaces tailored to augmented reality.

In augmented reality, two local users wearing head-mounted displays (HMDs) using general-purpose smartphones play the roles of a fox and the Little Prince respectively, while another remote user wearing an HMD in virtual reality plays the role of a scarecrow character. Through this, ETRI was able to demonstrate remote immersive extended reality involving multiple participants.

Meanwhile, this research was jointly conducted over four years by ETRI, KAIST, J2YSoft Co., Ltd., and TOZ Co., Ltd., supported by a project from the Ministry of Science and ICT.

This content was produced with the assistance of AI translation services.