Simulation

Having spent years working with engineering design, optimization, control using simulation and AI technologies, I have long envisioned a workflow where these powerful tools could collaborate autonomously under Engineer supervision. Today, Agentic AI is making this possible.

Had a nice week in the Japan office with Siemens and Altair colleagues, presenting at JSAE and a packed schedule of customer and partner visits.

When working with industrial AI, training a high performance prediction model is only one part of the challenges. Other critical aspects include:

AI Agents and Agentic AI are increasingly considered in engineering workflows, not just for coding, reasoning, but also to learn from historical data and enhance automation. The big question remains: what real added value do they bring to engineering practice, i.e how AI Agents helps to accelerate simulation modeling, control design, or MBSE processes.

As the robotics revolution accelerates, one of our key visions is to combine simulation with real world testing across all: from control systems to AI model development, to verification and validation (V&V)

The next wave of Industrial AI is accelerating, with physics-aware, trustworthy, and integration into engineering workflow.

It is a pleasure working with Prof. Alisa Rupenyan-Vasileva and the Postdoc Gabriele Fadini (ZHAW Zurich University of Applied Sciences), who are actively investigating state of the art autocalibration of high-dimensional control parameters using a combination of Digital Twin technology and AI methods.

Connecting real crash data with standards (Euro NCAP, UNECE, ISO…) and ASAM OpenSCENARIO model

I’m grateful to have participated Formula Student event this year again as a Judge for the driverless racing category, and delivered a lecture on recent Digital Twin and Generative AI technologies in the industry. Above all, it’s always a pleasure to see Siemens well presented in the event and on many of those cars :).

Nice pieces on suspension design, from 1D target cascading to 3D design requirements.

Bring Real data into Simulation (Real2Sim) and to... Movies.

Bridging the gap of Real and Simulation (Real2Sim - Sim2Real) toward engineering values is interesting and motivating to see how far we could push the boundary. Take a look at our 3D scene reconstruction demo in a parking scene, compare virtual and real side by side.

When reconstructing 3D traffic model from real world data (Real2Sim), one of the challenges is incorporating physical properties such as materials and textures.

Transform the vehicle sensor data into a large AI neural network model. The Gaussian Splatting model captures very well all details of the street from other cars, road terrains, trees, buildings… with high texture, color performance.

Nice joint research project that combines academic and industrial environments. You will work on state of art control technologies and some cool applications :).

Autocalibration control technologies can drastically reduce time and cost for engineers when moving from simulation to real-world testing. The method is for wide range of applications, not only autonomous driving.

While simulation continues to play an important role in autonomous driving development and validation, one of the main challenges is to have a credible simulation environment and data.

Generative AI opens capabilities to generate video. Still, the applications are mainly in entertainment, advertisement or similar, limited in engineering. It is still very hard to includes physics, structure, interaction, repeatability, temporal consistent, diversity…

Data is very valuable for autonomous driving/ADAS engineers, i.e. training & validation of ADAS functions such as perception, planning, control.

Together with Siemens colleagues in Japan, we are happy to jointly presenting our recent technologies on logged data and simulation exploitation for ADAS validation in JSAE2024 Congress (Japanese Society of Automotive Engineers), May 22-24 in Yokohama, Japan.

Data-driven control has become more popular recently, where measurement data is exploited to design higher performance controller, robust to uncertainties and change of environment.

Autonomous vehicle dataset such as nuScenes, Waymo Motion, or your own collected one, is known valuable for algorithm development and testing. What's more?

The goal of this project is to enhance safety and comfort in autonomous driving. Altogether we are developing and advancing different ADAS technologies, i.e. data collection, processing, scenario extraction-generation, algorithms (AI, control), XiL testing (X = model, hardware, human), and validation using several autonomous vehicle platforms.

While keep pushing the boundary of AI technologies in safety-critical applications like autonomous vehicles and medical applications, at Siemens we have been jointly develop with FOCETA partners technologies like requirements formulation, critical scenario generation, safety monitoring, explainable AI, adversarial examples....

Bring real collected data into simulation (Real2Sim) is essential activity in ADAS testing, allow engineers to test different sensor, vehicle dynamics models, as well as various what-if traffic scenarios.

This picture shows a Digital Twin vehicle together with the real Red Bull F1 vehicle (that Siemens partnering with), that has been inspiring us on developing Digital Twin for ADAS, autonomous driving.

ADAS or autonomous driving dataset is captured from vehicle sensors, and often used for perception algorithms or motion prediction. However, logged data is often passive, static, and open-loop. It is not able to actively adapt or evolve, for example with respect to the new sensor types. As the data is with human driver providing vehicle actions, closed-loop control testing is not possible.

Following the exciting Formula Student event last month, FSG and Siemens is organizing a workshop where we will show some tools, applications and algorithms around Digital Twin that can help you to develop autonomous driving functions more efficient. Check the below infor for more information.

See following demo from Siemens research engineer Jean Pierre Allamaa, done within the EU ELO-X Marie Curie project and in collaboration with Prof. Toshiyuki Ohtsuka (Kyoto Univ.) during a secondment program.

SADAS performance is often evaluated based on safety, and from the outsider's perspective, i.e. check if there is a collision. What is not commonly known is that perceived safety and comfort are similarly important in ADAS development. See below a short interesting demo from insider or passenger's view on lane change maneuvers. "

Glad to see quite some Digital Twin research activities from UK robotics and control peers in the workshop.

Learning from human driving demonstration is an interesting approach to improve autonomous driving policy. Still, it has some challenges: 1. majority logged data is from normal traffic situations, not sufficient critical and diverse scenarios data; and 2. during training and validation, the other traffic actors do not react to the ego's car policy, or no actual closed-loop response.

It is a nice way to start the weekend after a busy week: Our ADAS R&D team just received notifications that 3 papers got accepted to IFAC World Congress - (probably the biggest event of the Control system society this year). A nice achievement of the team members.

Reinforcement learning (RL) has shown capabilities to deal with complex systems, including autonomous driving. However, most results are only in simulation or game environments as training/testing in real life is unsafe and expensive.

with TU Berlin and University of Oxford. Thanks again Siemens RIE colleagues, Ilaria and Susanne to make this nice interview on our thoughts and sharing in digital twin of connected mobility (also for the nice pictures :) !

We are working together with other EU partners to develop continuous engineering of trustworthy autonomy and implement on industrial autonomous system use cases. In this project newsletter, you can discover more our recent findings and activities. In addition, I also gave an interview (page 7) on my role in the project and more general views on not only technical challenges but also collaborations between industry-academic

a self-contained virtual representation that models physical behaviors and combines with physical data to provide augmented information.

Significant efforts have been also devoted to building and tuning a large and high quality 3D Munich city model (on realism, structure, texture, material). The model is then optimized into a real-time VR platform in Unity, and also Simcenter Prescan to exploit physics-based sensors (lidar, camera, radar) and perception-control algorithms.

I will be speaking in the ADAS &AV Tech Expo next month in San Jose. Hope to see and exchange with you there.

I will be speaking in AutoSens, Brussels

Happy to share some of our team recent works and results on multiple topics around autonomous driving technologies development, leveraged by digital twin. The works are on both physical vehicle testing and virtual validation, implemented by research engineers and industrial PhDs in Engineering Services ADAS at Siemens Digital Industries Software and collaborators.)

We show a demo comparison for an autonomous lane change scenario, where the same vehicle control system was applied to two vehicle dynamics models

In the last few months, we have made some more progress to tackle some autonomous driving challenges in industry.

Testing and validation are big challenges in the autonomous vehicles driving industry. Your ADAS engineers keep developing new perception or control algorithms, but what is an efficient process starting from there to vehicle deployment?

People actually wants more from autonomous car, be cautious but also aggressive when necessary, be able and flexible moving along the tradeoff curves until physical limitation.

Besides, a nice example of collaboration not only within the team engineers but also with Marketing dept., who captures well the main messages, on-site ADAS demos and put them in such illustrative video

Next challenge is having an efficient framework from simulation to physical testing to validate your designs. This demo shows an example of our team works on MPC control development.

Vehicle dynamics is essential for autonomous driving, in both safety and comfort performance. We show how to build a high fidelity vehicle model via simulation and proving ground testing, then exploit it for a safety-critical autonomous double lane change optimal control (MPC) development. This is a great video, with interesting (autonomous) driving scenes. Hope you will enjoy!

Several interesting advanced control technologies will be discussed there: MPC, combined model-based and AI or data learning from both human data or model learning, reinforcement learning, drift parking control, truck crossing roundabout with a failed steering, handling in snowy weather, virtual sensing, verification of NN, and also XiL testing

A nice work and interesting whitepaper from Siemens DI colleagues on autonomous driving scenarios testing, verification and validation. These topics are critical for your autonomous vehicle development

I'm proud to have received the 2018 Siemens DF PL Invention of the Year Award. The solution will be contributed to Siemens PLM Software autonomous vehicles sector business

A great pleasure to visit Automatic Control Laboratory (LA) and Prof. Colin Jones