SoftBank : How SoftBank Corp. and Keio Research Institute at SFC are Using Digital Twin to Enhance Autonomous Shuttle Bus Operations

Keio Research Institute at SFC (KRIS) and SoftBank Corp. (TOKYO: 9434)'s Research Institute of Advanced Technology are collaborating on a new project that may serve as a model for automated driving. In May 2023, SoftBank and KRIS announced they launched a proof of concept (PoC) to enhance the operation of autonomous shuttle buses at Keio University Shonan Fujisawa Campus (SFC), which is located in Fujisawa City south of Tokyo. The PoC is unique in that it utilizes Digital Twin, a technology that reproduces a digital replica of the physical world.

Digital Twin the key to advanced automated driving

Key to this PoC is the Digital Twin Campus Lab established by KRIS and SoftBank in 2022. On a campus with a standalone* 5G network, information and events in the physical world will be digitized and shared in a virtual space using image recognition and spatial sensing. Various research and development projects are underway using this Digital Twin Platform, and a PoC for the advanced operation of autonomous shuttle buses is one of them.

• *

  Standalone: an advanced wireless technology that combines new 5G dedicated core facilities and 5G base stations, unlike a conventional non-standalone system that employs a structure that couples legacy 4G core network equipment with 5G base stations.

View of the SFC Campus rendered by the Digital Twin Platform developed by the Digital Twin Campus Lab

In addition to in-vehicle cameras and sensors, six sensors installed around the campus are now acquiring information, while cameras are capturing traffic light signal data. While verifying how these data sets can be utilized with the Digital Twin, SoftBank and KRIS are applying them to the practical advancement of automated driving.

Numerous controls required for autonomous driving

In this PoC, two things are being put to the test: (1) the detection of oncoming vehicles while making right turns and (2) traffic signal prediction.

(1) Detection of oncoming vehicles while making right turns
SFC's driving route is a mixture of straight and right-turn lanes with curves. To make automated driving more sophisticated, data on surroundings is key. For example, the status of oncoming vehicles coming from a distance cannot be detected just by cameras mounted on vehicles. By receiving information from the Digital Twin platform, however, the autonomous shuttle bus system can assess traffic situations well in advance from a distance. It can also receive data on how many people are present, how they are moving and whether vehicles are approaching from afar. This enables more advanced automated driving and hazard avoidance.

In the past, right turns were made manually by the driver after making visual checks, but in this PoC, tests are being conducted to switch to automatic right-turn operation only when safety is confirmed by acquiring Digital Twin-reproduced information on other vehicles and pedestrians, and by ascertaining the presence of faraway oncoming vehicles that cannot be detected by the vehicle's onboard sensors.

Screen displayed on a monitor inside automated bus

(2) Traffic signal prediction
With automated driving, it is critical for the vehicle to know when traffic lights turn red or green to ensure ride comfort and safety. For example, if it is possible to predict when a traffic light will turn red, deceleration timing can be automatically adjusted to prevent sudden braking. Furthermore, if past traffic light data can be analyzed to predict how many seconds it will take for a traffic light to change, the vehicle can decelerate in advance to avoid braking before an intersection.

Cameras installed in vehicles sometimes fail to detect traffic signal information due to backlighting, among other reasons. In this PoC trial, AI is being used to estimate traffic light information based on signal images taken in the SFC vicinity, and this information is shared with autonomous shuttle buses.

Cameras installed in SFC buildings acquire traffic light information

Press Release: SoftBank Corp. and Keio Research Institute at SFC Commence PoC for the Advanced Operation of Autonomous Shuttle Buses Utilizing Digital Twin at Digital Twin Campus Lab

SoftBank Corp. Research Institute of Advanced Technology Stories: Efforts to Enhance Autonomous Shuttle Bus Operations Using Digital Twins

(Posted on July 18, 2023, Original article posted on June 19, 2023)
by SoftBank News Editors