Seeing around the traffic, we can recognize easily all objects, cars, pedestrians around, and more than that, know the type of road, weather, street… Seeing the clouds to realize rain is coming soon (and anticipate some heavy traffic…). Develop AI models to sense all these tasks is hard and expensive.
This is now potential with large vision-language model which is capable of reasoning from image data. Still, how this can combine with other important safety and ‘engineering’ metrics like time to collision, distance to objects… Our ADAS team at Siemens Digital Industries Software has been working on new Generative AI technologies with those angles in mind. We developed an unified tool to have a full understanding of a traffic scenario: from semantic information, reasoning to safety metrics. It is beneficial for many ADAS testing, requirements, perception, and safety engineers.
A nice demo that you may enjoy watching it as myself…
It is realized with a great implementation contribution from Samuele Ruffino, a talent AI intern from ETH Zürich working in our team.