Autonomous Systems

A study of human-like autonomous driving controllers in a driving simulator just got published in Transportation Research Part F: Traffic Psychology and Behavior. The article is written by Flavia Sofia Acerbo, the industrial PhD at Siemens and KU Leuven

As the robotics revolution accelerates, one of our key visions is to combine simulation with real world testing across all: from control systems to AI model development, to verification and validation (V&V)

Last week, I had the pleasure visiting the Oxford Control Group to give a seminar and exchange ideas with faculty members and students. It was also great to meet the PhD student Rory Halsall, whom we jointly supervise within the collaboration between UK Research and Innovation and Siemens :).

It is great to welcome partners and colleagues from the EU RobustifAI project (Robustifying Generative AI through Human-centric integration of Neural and Symbolic methods) to Siemens office for technical updates, discussions, and AI knowledge exchange.

In recent months, I’ve had the great honor of serving on PhD Defense Juries of four PhDs:

The nice image comes straight from the actual research & development results. From right to left, you can see the Gaussian Splatting training process progressively improve the 3D reconstruction.

I’m grateful to have participated Formula Student event this year again as a Judge for the driverless racing category, and delivered a lecture on recent Digital Twin and Generative AI technologies in the industry. Above all, it’s always a pleasure to see Siemens well presented in the event and on many of those cars :).

FYI, interesting PhD position on data-driven optimal control for Air Mobility and Battery Management in Oxford Control Group.

Launched in June 2025 with a total budget of 9.3M€. The project aims to develop a rigorous design and deployment methodology tailored for reliable, robust and trustworthy Generative AI.

Bring Real data into Simulation (Real2Sim) and to... Movies.

Bridging the gap of Real and Simulation (Real2Sim - Sim2Real) toward engineering values is interesting and motivating to see how far we could push the boundary. Take a look at our 3D scene reconstruction demo in a parking scene, compare virtual and real side by side.

One often overlooked topic in control systems engineering is explainability. Control engineers are typically trained, through university or research, to design systems from the ground up, starting with physics-based modeling, model identification, stability analysis, control design, and then iterating with performance evaluation. This process provides engineers insight into the system, enabling intuitive for parameter calibration.

When reconstructing 3D traffic model from real world data (Real2Sim), one of the challenges is incorporating physical properties such as materials and textures.

I am honored to have opportunity to speak at this distinguished gathering alongside experts in AI and industry

Great discussions with Prof. Holger Caesar (Intelligent Vehicles Group, TU Delft) with his visiting to our R&D team in Siemens Leuven office last Friday.

Transform the vehicle sensor data into a large AI neural network model. The Gaussian Splatting model captures very well all details of the street from other cars, road terrains, trees, buildings… with high texture, color performance.

Driving from Vision through Differentiable Optimal Control

In the coming days, I will be participating the two EU forums related to AI, robotics and autonomous driving.

A great pleasure to have our two research engineers going to present the company Siemens Digital Industries Software R&D activities on human-centric AI in IEEE IROS and InCabin conferences this month.

From my industrial PhD Jean-Pierre: Proud and honored to have won the Young Author Award at the 8th International Federation of Automatic Control Nonlinear Model Predictive (IFAC NMPC2024) conference, held in the prestigious city of Kyoto, Japan.

While simulation continues to play an important role in autonomous driving development and validation, one of the main challenges is to have a credible simulation environment and data.

Generative AI opens capabilities to generate video. Still, the applications are mainly in entertainment, advertisement or similar, limited in engineering. It is still very hard to includes physics, structure, interaction, repeatability, temporal consistent, diversity…

Pleasure to have Siemens and our activities on autonomous driving being shown in the L4DC conference

Data is very valuable for autonomous driving/ADAS engineers, i.e. training & validation of ADAS functions such as perception, planning, control.

I believe not only me but many autonomous driving engineers/researchers once has a dream that one day the vehicle sensor can sense the environment like we human do.

Together with Siemens colleagues in Japan, we are happy to jointly presenting our recent technologies on logged data and simulation exploitation for ADAS validation in JSAE2024 Congress (Japanese Society of Automotive Engineers), May 22-24 in Yokohama, Japan.

Getting more insights of human driving preferences (i.e. when and how to brake, steering at different traffic scenarios), perceived safety, different cultures and styles,... are essential to improve ADAS control development

Last week, I had the pleasure of visiting Oxford Control Group and deliver a seminar there. It was an excellent opportunity to meet and discuss with great researchers in control systems, data-driven, and safety-based optimization fields. I was also delighted to hear that our works have inspired some PhDs there on their research :) .

4-5 years ago we developed an intuition to deal with this challenge, learning from control barrier function (CBF) from legged robotics community.

Autonomous vehicle dataset such as nuScenes, Waymo Motion, or your own collected one, is known valuable for algorithm development and testing. What's more?

The goal of this project is to enhance safety and comfort in autonomous driving. Altogether we are developing and advancing different ADAS technologies, i.e. data collection, processing, scenario extraction-generation, algorithms (AI, control), XiL testing (X = model, hardware, human), and validation using several autonomous vehicle platforms.

While keep pushing the boundary of AI technologies in safety-critical applications like autonomous vehicles and medical applications, at Siemens we have been jointly develop with FOCETA partners technologies like requirements formulation, critical scenario generation, safety monitoring, explainable AI, adversarial examples....

Bring real collected data into simulation (Real2Sim) is essential activity in ADAS testing, allow engineers to test different sensor, vehicle dynamics models, as well as various what-if traffic scenarios.

This semester we have a pleasure to welcome two Master thesis students working full time with our ADAS team in Siemens Digital Industries, Leuven office: Jasper van Leuven (TUDelft, Netherlands) and Sven Becker (EPFL, Switzerland).

This picture shows a Digital Twin vehicle together with the real Red Bull F1 vehicle (that Siemens partnering with), that has been inspiring us on developing Digital Twin for ADAS, autonomous driving.

ADAS or autonomous driving dataset is captured from vehicle sensors, and often used for perception algorithms or motion prediction. However, logged data is often passive, static, and open-loop. It is not able to actively adapt or evolve, for example with respect to the new sensor types. As the data is with human driver providing vehicle actions, closed-loop control testing is not possible.

Today marks my 7-year PhD graduation from KU Leuven as well as the time working at Siemens Digital Industries Software. My experiences and ambitions of innovation keep rolling :).

Following the exciting Formula Student event last month, FSG and Siemens is organizing a workshop where we will show some tools, applications and algorithms around Digital Twin that can help you to develop autonomous driving functions more efficient. Check the below infor for more information.

See following demo from Siemens research engineer Jean Pierre Allamaa, done within the EU ELO-X Marie Curie project and in collaboration with Prof. Toshiyuki Ohtsuka (Kyoto Univ.) during a secondment program.

SADAS performance is often evaluated based on safety, and from the outsider's perspective, i.e. check if there is a collision. What is not commonly known is that perceived safety and comfort are similarly important in ADAS development. See below a short interesting demo from insider or passenger's view on lane change maneuvers. "

1. Reinforcement Learning from Simulation to Real World; 2. MPC-Based Imitation Learning for Human-Like Autonomous Driving; 3. Critical Driving Behaviours Using Driver's Risk Field

Data-driven control: advanced optimal control, imitation learning, driver behavior models, reinforcement learning

A great pleasure to act as a judge for the interesting Formula Student Germany event this week, in the driverless category. Seeing many talented, motivated students working and speaking on similar technologies to what we are doing in ADAS industry ;).

It was exciting to welcome the MECO group from KU Leuven last Friday at Siemens Digital Industries Software in Leuven.

This period we have had a pleasure to welcome two visiting PhDs: Shuhao Zhang and Renzi Wang from KULeuven, who are visiting our ADAS team at Siemens Digital Industries Software office in Leuven for 3-4 months. Both is working on ADAS challenges related to learning perception and motion planning under uncertainty conditions, with driving data from Siemens.

Learning from human driving demonstration is an interesting approach to improve autonomous driving policy. Still, it has some challenges: 1. majority logged data is from normal traffic situations, not sufficient critical and diverse scenarios data; and 2. during training and validation, the other traffic actors do not react to the ego's car policy, or no actual closed-loop response.

It is a nice way to start the weekend after a busy week: Our ADAS R&D team just received notifications that 3 papers got accepted to IFAC World Congress - (probably the biggest event of the Control system society this year). A nice achievement of the team members.

Reinforcement learning (RL) has shown capabilities to deal with complex systems, including autonomous driving. However, most results are only in simulation or game environments as training/testing in real life is unsafe and expensive.

This paper draft is a nice piece of writing with quite some works, discussions, demonstrations, and evaluations on different learning methods.

with TU Berlin and University of Oxford. Thanks again Siemens RIE colleagues, Ilaria and Susanne to make this nice interview on our thoughts and sharing in digital twin of connected mobility (also for the nice pictures :) !

We are working together with other EU partners to develop continuous engineering of trustworthy autonomy and implement on industrial autonomous system use cases. In this project newsletter, you can discover more our recent findings and activities. In addition, I also gave an interview (page 7) on my role in the project and more general views on not only technical challenges but also collaborations between industry-academic

Significant efforts have been also devoted to building and tuning a large and high quality 3D Munich city model (on realism, structure, texture, material). The model is then optimized into a real-time VR platform in Unity, and also Simcenter Prescan to exploit physics-based sensors (lidar, camera, radar) and perception-control algorithms.

I will be speaking in the ADAS &AV Tech Expo next month in San Jose. Hope to see and exchange with you there.

I will be speaking in AutoSens, Brussels

The industrial PhD of Flavia is tackling some very interesting research, and this nice Award apparently shows a concrete progress. The work will also be presented at ICML (International Conference on Machine Learning) next week in Baltimore.

This week we just organized a workshop in Leuven office for the Marie Curie ITN ELO-X supervisors and ESR fellows.

Happy to share some of our team recent works and results on multiple topics around autonomous driving technologies development, leveraged by digital twin. The works are on both physical vehicle testing and virtual validation, implemented by research engineers and industrial PhDs in Engineering Services ADAS at Siemens Digital Industries Software and collaborators.)

An interesting interview on the topic of ADAS comfort, made by Siemens Simcenter Engineering with our ADAS research engineer, Flavia.

Today I got a great pleasure to be an invited industrial speaker to the students of the Automation and Control Engineering Program of Politecnico di Milano, one of the largest programs I know so far in the field (i.e. with more than 400 control engineering students). I was excited with the opportunity to share and inspire the students with Siemens Simcenter, Digital Twin and engineering technologies, in particular our team developing autonomous vehicle control and testing solutions. Thank you again Prof. Lorenzo Fagiano for the invitation!

We present the development and deployment of an embedded optimal control strategy for autonomous driving applications on a Ford Focus road vehicle.Non-linear model predictive control (NMPC) is designed and deployed on a system with hard real-time constraints. We show the properties of sequential quadratic programming (SQP) optimization solvers that are suitable for driving tasks

We show a demo comparison for an autonomous lane change scenario, where the same vehicle control system was applied to two vehicle dynamics models

Covid time and new rules make it harder for both companies and universities. Still, this semester our R&D team at Siemens Digital Industries Software just welcomed three Master thesis/intern students coming from TU Delft Master of Robotics: Kevin Voogd, Jianfeng Cui, and Yurui Du.

In the last few months, we have made some more progress to tackle some autonomous driving challenges in industry.

Testing and validation are big challenges in the autonomous vehicles driving industry. Your ADAS engineers keep developing new perception or control algorithms, but what is an efficient process starting from there to vehicle deployment?

The key is to optimize control actions over a prediction horizon, which is often short due to computation.

This is my third year serving in the control system society as Associate Editor for conferences like CDC and ACC. A small role but I find this editorial work is an interesting way to keep updated with (and learn from) latest developments regularly from academics, and at the same time provide my R&D industry visions to the community!

Great to hear that. Looking back I have supervised some students doing thesis/internship in the company in the last 2-3 years, and six of them found their profesional careers after study at different teams of Siemens.

People actually wants more from autonomous car, be cautious but also aggressive when necessary, be able and flexible moving along the tradeoff curves until physical limitation.

Besides, a nice example of collaboration not only within the team engineers but also with Marketing dept., who captures well the main messages, on-site ADAS demos and put them in such illustrative video

Next challenge is having an efficient framework from simulation to physical testing to validate your designs. This demo shows an example of our team works on MPC control development.

If you are participating and interested in automotive & autonomous vehicle, please consider the automotive control session next Wednesday afternoon. I and Karl Berntorp will co-chair it, hopefully we would have some interesting discussions and chats there.

Vehicle dynamics is essential for autonomous driving, in both safety and comfort performance. We show how to build a high fidelity vehicle model via simulation and proving ground testing, then exploit it for a safety-critical autonomous double lane change optimal control (MPC) development. This is a great video, with interesting (autonomous) driving scenes. Hope you will enjoy!

Several interesting advanced control technologies will be discussed there: MPC, combined model-based and AI or data learning from both human data or model learning, reinforcement learning, drift parking control, truck crossing roundabout with a failed steering, handling in snowy weather, virtual sensing, verification of NN, and also XiL testing

With the CCTA accepted paper, we are happy to be involved and show our innovative solutions in the recent four largest events of IFAC and IEEE Control System Society.

A nice work and interesting whitepaper from Siemens DI colleagues on autonomous driving scenarios testing, verification and validation. These topics are critical for your autonomous vehicle development

It is my great pleasure to be in the finalist of AutoSens Award in the Most Influential Research Category for the our R&D works on autonomous vehicle control developments!

I'm proud to have received the 2018 Siemens DF PL Invention of the Year Award. The solution will be contributed to Siemens PLM Software autonomous vehicles sector business

A great pleasure to visit Automatic Control Laboratory (LA) and Prof. Colin Jones