Digital Twin

Having spent years working with engineering design, optimization, control using simulation and AI technologies, I have long envisioned a workflow where these powerful tools could collaborate autonomously under Engineer supervision. Today, Agentic AI is making this possible.

Had a nice week in the Japan office with Siemens and Altair colleagues, presenting at JSAE and a packed schedule of customer and partner visits.

When working with industrial AI, training a high performance prediction model is only one part of the challenges. Other critical aspects include:

AI Agents and Agentic AI are increasingly considered in engineering workflows, not just for coding, reasoning, but also to learn from historical data and enhance automation. The big question remains: what real added value do they bring to engineering practice, i.e how AI Agents helps to accelerate simulation modeling, control design, or MBSE processes.

As the robotics revolution accelerates, one of our key visions is to combine simulation with real world testing across all: from control systems to AI model development, to verification and validation (V&V)

Last week, I had the pleasure visiting the Oxford Control Group to give a seminar and exchange ideas with faculty members and students. It was also great to meet the PhD student Rory Halsall, whom we jointly supervise within the collaboration between UK Research and Innovation and Siemens :).

The next wave of Industrial AI is accelerating, with physics-aware, trustworthy, and integration into engineering workflow.

The nice image comes straight from the actual research & development results. From right to left, you can see the Gaussian Splatting training process progressively improve the 3D reconstruction.

It is a pleasure working with Prof. Alisa Rupenyan-Vasileva and the Postdoc Gabriele Fadini (ZHAW Zurich University of Applied Sciences), who are actively investigating state of the art autocalibration of high-dimensional control parameters using a combination of Digital Twin technology and AI methods.

I’m grateful to have participated Formula Student event this year again as a Judge for the driverless racing category, and delivered a lecture on recent Digital Twin and Generative AI technologies in the industry. Above all, it’s always a pleasure to see Siemens well presented in the event and on many of those cars :).

FYI, interesting PhD position on data-driven optimal control for Air Mobility and Battery Management in Oxford Control Group.

Nice pieces on suspension design, from 1D target cascading to 3D design requirements.

Launched in June 2025 with a total budget of 9.3M€. The project aims to develop a rigorous design and deployment methodology tailored for reliable, robust and trustworthy Generative AI.

Bring Real data into Simulation (Real2Sim) and to... Movies.

Bridging the gap of Real and Simulation (Real2Sim - Sim2Real) toward engineering values is interesting and motivating to see how far we could push the boundary. Take a look at our 3D scene reconstruction demo in a parking scene, compare virtual and real side by side.

One often overlooked topic in control systems engineering is explainability. Control engineers are typically trained, through university or research, to design systems from the ground up, starting with physics-based modeling, model identification, stability analysis, control design, and then iterating with performance evaluation. This process provides engineers insight into the system, enabling intuitive for parameter calibration.

When reconstructing 3D traffic model from real world data (Real2Sim), one of the challenges is incorporating physical properties such as materials and textures.

I am honored to have opportunity to speak at this distinguished gathering alongside experts in AI and industry

Transform the vehicle sensor data into a large AI neural network model. The Gaussian Splatting model captures very well all details of the street from other cars, road terrains, trees, buildings… with high texture, color performance.

Nice joint research project that combines academic and industrial environments. You will work on state of art control technologies and some cool applications :).

Driving from Vision through Differentiable Optimal Control

In the coming days, I will be participating the two EU forums related to AI, robotics and autonomous driving.

From my industrial PhD Jean-Pierre: Proud and honored to have won the Young Author Award at the 8th International Federation of Automatic Control Nonlinear Model Predictive (IFAC NMPC2024) conference, held in the prestigious city of Kyoto, Japan.

While simulation continues to play an important role in autonomous driving development and validation, one of the main challenges is to have a credible simulation environment and data.

Generative AI opens capabilities to generate video. Still, the applications are mainly in entertainment, advertisement or similar, limited in engineering. It is still very hard to includes physics, structure, interaction, repeatability, temporal consistent, diversity…

Data is very valuable for autonomous driving/ADAS engineers, i.e. training & validation of ADAS functions such as perception, planning, control.

Together with Siemens colleagues in Japan, we are happy to jointly presenting our recent technologies on logged data and simulation exploitation for ADAS validation in JSAE2024 Congress (Japanese Society of Automotive Engineers), May 22-24 in Yokohama, Japan.

Data-driven control has become more popular recently, where measurement data is exploited to design higher performance controller, robust to uncertainties and change of environment.

Getting more insights of human driving preferences (i.e. when and how to brake, steering at different traffic scenarios), perceived safety, different cultures and styles,... are essential to improve ADAS control development

Autonomous vehicle dataset such as nuScenes, Waymo Motion, or your own collected one, is known valuable for algorithm development and testing. What's more?

Bring real collected data into simulation (Real2Sim) is essential activity in ADAS testing, allow engineers to test different sensor, vehicle dynamics models, as well as various what-if traffic scenarios.

This semester we have a pleasure to welcome two Master thesis students working full time with our ADAS team in Siemens Digital Industries, Leuven office: Jasper van Leuven (TUDelft, Netherlands) and Sven Becker (EPFL, Switzerland).

This picture shows a Digital Twin vehicle together with the real Red Bull F1 vehicle (that Siemens partnering with), that has been inspiring us on developing Digital Twin for ADAS, autonomous driving.

ADAS or autonomous driving dataset is captured from vehicle sensors, and often used for perception algorithms or motion prediction. However, logged data is often passive, static, and open-loop. It is not able to actively adapt or evolve, for example with respect to the new sensor types. As the data is with human driver providing vehicle actions, closed-loop control testing is not possible.

Today marks my 7-year PhD graduation from KU Leuven as well as the time working at Siemens Digital Industries Software. My experiences and ambitions of innovation keep rolling :).

Following the exciting Formula Student event last month, FSG and Siemens is organizing a workshop where we will show some tools, applications and algorithms around Digital Twin that can help you to develop autonomous driving functions more efficient. Check the below infor for more information.

See following demo from Siemens research engineer Jean Pierre Allamaa, done within the EU ELO-X Marie Curie project and in collaboration with Prof. Toshiyuki Ohtsuka (Kyoto Univ.) during a secondment program.

SADAS performance is often evaluated based on safety, and from the outsider's perspective, i.e. check if there is a collision. What is not commonly known is that perceived safety and comfort are similarly important in ADAS development. See below a short interesting demo from insider or passenger's view on lane change maneuvers. "

A great pleasure to act as a judge for the interesting Formula Student Germany event this week, in the driverless category. Seeing many talented, motivated students working and speaking on similar technologies to what we are doing in ADAS industry ;).

This period we have had a pleasure to welcome two visiting PhDs: Shuhao Zhang and Renzi Wang from KULeuven, who are visiting our ADAS team at Siemens Digital Industries Software office in Leuven for 3-4 months. Both is working on ADAS challenges related to learning perception and motion planning under uncertainty conditions, with driving data from Siemens.

Glad to see quite some Digital Twin research activities from UK robotics and control peers in the workshop.

Learning from human driving demonstration is an interesting approach to improve autonomous driving policy. Still, it has some challenges: 1. majority logged data is from normal traffic situations, not sufficient critical and diverse scenarios data; and 2. during training and validation, the other traffic actors do not react to the ego's car policy, or no actual closed-loop response.

It is a nice way to start the weekend after a busy week: Our ADAS R&D team just received notifications that 3 papers got accepted to IFAC World Congress - (probably the biggest event of the Control system society this year). A nice achievement of the team members.

Reinforcement learning (RL) has shown capabilities to deal with complex systems, including autonomous driving. However, most results are only in simulation or game environments as training/testing in real life is unsafe and expensive.

with TU Berlin and University of Oxford. Thanks again Siemens RIE colleagues, Ilaria and Susanne to make this nice interview on our thoughts and sharing in digital twin of connected mobility (also for the nice pictures :) !

We are working together with other EU partners to develop continuous engineering of trustworthy autonomy and implement on industrial autonomous system use cases. In this project newsletter, you can discover more our recent findings and activities. In addition, I also gave an interview (page 7) on my role in the project and more general views on not only technical challenges but also collaborations between industry-academic

a self-contained virtual representation that models physical behaviors and combines with physical data to provide augmented information.

Significant efforts have been also devoted to building and tuning a large and high quality 3D Munich city model (on realism, structure, texture, material). The model is then optimized into a real-time VR platform in Unity, and also Simcenter Prescan to exploit physics-based sensors (lidar, camera, radar) and perception-control algorithms.

I will be speaking in the ADAS &AV Tech Expo next month in San Jose. Hope to see and exchange with you there.

I will be speaking in AutoSens, Brussels

Happy to share some of our team recent works and results on multiple topics around autonomous driving technologies development, leveraged by digital twin. The works are on both physical vehicle testing and virtual validation, implemented by research engineers and industrial PhDs in Engineering Services ADAS at Siemens Digital Industries Software and collaborators.)

Today I got a great pleasure to be an invited industrial speaker to the students of the Automation and Control Engineering Program of Politecnico di Milano, one of the largest programs I know so far in the field (i.e. with more than 400 control engineering students). I was excited with the opportunity to share and inspire the students with Siemens Simcenter, Digital Twin and engineering technologies, in particular our team developing autonomous vehicle control and testing solutions. Thank you again Prof. Lorenzo Fagiano for the invitation!

We present the development and deployment of an embedded optimal control strategy for autonomous driving applications on a Ford Focus road vehicle.Non-linear model predictive control (NMPC) is designed and deployed on a system with hard real-time constraints. We show the properties of sequential quadratic programming (SQP) optimization solvers that are suitable for driving tasks

We show a demo comparison for an autonomous lane change scenario, where the same vehicle control system was applied to two vehicle dynamics models

In the last few months, we have made some more progress to tackle some autonomous driving challenges in industry.

Testing and validation are big challenges in the autonomous vehicles driving industry. Your ADAS engineers keep developing new perception or control algorithms, but what is an efficient process starting from there to vehicle deployment?

People actually wants more from autonomous car, be cautious but also aggressive when necessary, be able and flexible moving along the tradeoff curves until physical limitation.

Besides, a nice example of collaboration not only within the team engineers but also with Marketing dept., who captures well the main messages, on-site ADAS demos and put them in such illustrative video

Vehicle dynamics is essential for autonomous driving, in both safety and comfort performance. We show how to build a high fidelity vehicle model via simulation and proving ground testing, then exploit it for a safety-critical autonomous double lane change optimal control (MPC) development. This is a great video, with interesting (autonomous) driving scenes. Hope you will enjoy!

With the CCTA accepted paper, we are happy to be involved and show our innovative solutions in the recent four largest events of IFAC and IEEE Control System Society.

A nice work and interesting whitepaper from Siemens DI colleagues on autonomous driving scenarios testing, verification and validation. These topics are critical for your autonomous vehicle development

I'm proud to have received the 2018 Siemens DF PL Invention of the Year Award. The solution will be contributed to Siemens PLM Software autonomous vehicles sector business

A great pleasure to visit Automatic Control Laboratory (LA) and Prof. Colin Jones