Rakuten Mobile

Do you have a project in mind that you can tell me about that you used Immersal in?

The most recent project that incorporated the use of Immersal technology was our Mixed Reality Racing project. We wanted to create a new kind of racing experience fusing real and virtual racing.

Motorsport racing has a long history of drivers competing against each other on real race tracks with real racing cars. But in recent years, interest in esports has been growing tremendously, with ever increasing prize money, audience numbers, and professional players and sponsored teams. These e-sports players can compete on racing simulators side-by-side or remotely on the same game. We were fascinated by the idea of enabling e-racers to race against real drivers on a race track at the same time, connecting the two types of racing together.




I don't think anyone has ever done this before. And now with 5G and the capability of streaming large amount of data in real-time, I think it's about time we make this dream a reality. We conducted a successful demonstration by using real time synchronous communication, synchronizing telemetry data from within the real car and data from within the game. So the two are synchronized in real time using 4G and 5G – allowing the professional e-sports driver to compete with the professional racing driver in their respective machines! 

The demonstration was held during the 2022 Inter Proto Series POWERED BY KeePer Round 2 race at Fuji Speedway located in Shizuoka, Japan. One of the real race cars was imported into the game, which was represented as a virtual car, with the same colours. At the same time, the virtual game car is exported to the real world. From the grandstand at the racetrack, spectators could not only see the real race car, but also see the virtual race car through real-time synchronous AR displays. This was very exciting – to be able to see both cars side by side traveling at speeds of more than 240 km/hr along the main straight!!

What challenge did Immersal tech solve? 

Immersal tech enabled us to create large scale VPS maps of the facility and structure. It also let us localize the user and place AR content within the space, with streaming videos embedded into the physical environment. We were able to position the virtual AR car at the precise location on the racetrack in real time, synchronized with the virtual race car from the game.

In the future, with the use of 5G, we will be able to have many users within the same venue – including a large number of spectators in the grandstand, watch the same mixed reality racing experience at the same time. 

I think a lot of technologies only allow relatively small-scale mapping and localization which makes outdoor mapping a huge challenge. At the Fuji Speedway circuit, the grandstand is around 500m wide, while the main straight is around 1.5km long. So, in order to localize in such a large open space, you have to look at the buildings and structures around you. Immersal helped us do that, enabling us to provide an exciting new AR experience that combines both real and virtual motor sports racing. 

More details about the project in Rakuten Mobile's press release.