Perception

The nice image comes straight from the actual research & development results. From right to left, you can see the Gaussian Splatting training process progressively improve the 3D reconstruction.

Bring Real data into Simulation (Real2Sim) and to... Movies.

Bridging the gap of Real and Simulation (Real2Sim - Sim2Real) toward engineering values is interesting and motivating to see how far we could push the boundary. Take a look at our 3D scene reconstruction demo in a parking scene, compare virtual and real side by side.

When reconstructing 3D traffic model from real world data (Real2Sim), one of the challenges is incorporating physical properties such as materials and textures.

Great discussions with Prof. Holger Caesar (Intelligent Vehicles Group, TU Delft) with his visiting to our R&D team in Siemens Leuven office last Friday.

Transform the vehicle sensor data into a large AI neural network model. The Gaussian Splatting model captures very well all details of the street from other cars, road terrains, trees, buildings… with high texture, color performance.

Driving from Vision through Differentiable Optimal Control

While simulation continues to play an important role in autonomous driving development and validation, one of the main challenges is to have a credible simulation environment and data.

Data is very valuable for autonomous driving/ADAS engineers, i.e. training & validation of ADAS functions such as perception, planning, control.

I believe not only me but many autonomous driving engineers/researchers once has a dream that one day the vehicle sensor can sense the environment like we human do.

Autonomous vehicle dataset such as nuScenes, Waymo Motion, or your own collected one, is known valuable for algorithm development and testing. What's more?

Bring real collected data into simulation (Real2Sim) is essential activity in ADAS testing, allow engineers to test different sensor, vehicle dynamics models, as well as various what-if traffic scenarios.

ADAS or autonomous driving dataset is captured from vehicle sensors, and often used for perception algorithms or motion prediction. However, logged data is often passive, static, and open-loop. It is not able to actively adapt or evolve, for example with respect to the new sensor types. As the data is with human driver providing vehicle actions, closed-loop control testing is not possible.

This period we have had a pleasure to welcome two visiting PhDs: Shuhao Zhang and Renzi Wang from KULeuven, who are visiting our ADAS team at Siemens Digital Industries Software office in Leuven for 3-4 months. Both is working on ADAS challenges related to learning perception and motion planning under uncertainty conditions, with driving data from Siemens.

Significant efforts have been also devoted to building and tuning a large and high quality 3D Munich city model (on realism, structure, texture, material). The model is then optimized into a real-time VR platform in Unity, and also Simcenter Prescan to exploit physics-based sensors (lidar, camera, radar) and perception-control algorithms.

I will be speaking in AutoSens, Brussels

Covid time and new rules make it harder for both companies and universities. Still, this semester our R&D team at Siemens Digital Industries Software just welcomed three Master thesis/intern students coming from TU Delft Master of Robotics: Kevin Voogd, Jianfeng Cui, and Yurui Du.

Besides, a nice example of collaboration not only within the team engineers but also with Marketing dept., who captures well the main messages, on-site ADAS demos and put them in such illustrative video

It is my great pleasure to be in the finalist of AutoSens Award in the Most Influential Research Category for the our R&D works on autonomous vehicle control developments!