Intempora’s Post

Lambda is seriously cool, and I’m grateful for this official release… ML has always been a big part of the Nuro Driver, and our incredible team is constantly evaluating and trying new things. If you’re interested in the tech, check our blog post at https://lnkd.in/eGwbR5bn ! I’ve seen a company or two in this space proclaiming that using LLMs to be a new thing for autonomous driving, but we’ve been quietly evaluating and integrating them into our systems, rigorously evaluating them and testing their capabilities for some time now. The core tenant at Nuro is safety; we simply won’t implement new techniques or technologies into Nuro Driver until they have been proven safe and effective; the stakes are too high. However nice the headlines of proclaiming ourselves the “first” might be, if it means risking safety we simply won’t do it.

Diving Deeper into LAMBDA: The Nuro Driver's Real-Time Language Reasoning Model 🚗💬 Following our recent announcement of LAMBDA, our Multimodal Large Language Model (MLLM) integrated into The Nuro Driver's™ onboard autonomy stack, we're excited to share a deeper look at this groundbreaking technology in our latest technical blog post: https://lnkd.in/eGwbR5bn The blog post explores LAMBDA’s remarkable capabilities in understanding and reasoning about the diverse situations encountered by L4 driverless systems. Here at Nuro we believe MLLMs have the potential to revolutionize autonomous driving, and LAMBDA’s integration with our in-vehicle autonomy system is an important milestone towards that. LAMBDA also enables passengers to interact with our autonomy system through an in-vehicle tablet interface, answering their questions and providing commentary on the world around them and the autonomous vehicle’s driving decisions. Read the blog post to learn more about LAMBDA's architecture and the challenges we're overcoming to utilize this powerful language model for real-time decision making. Join us as we continue to push the boundaries of autonomous driving and shape the future of mobility. Explore career opportunities at Nuro: https://lnkd.in/gUNJ_zsh #AutonomousDriving #MLLM #LanguageModels #Mobility #Innovation #Careers

Without some method of language integration, scaling self-driving is more difficult and requires larger amounts of driving data. In contrast, a human who has learned driving in San Francisco doesn’t really need new driving experience in New York City, just to learn the local rule that you shouldn’t make a right turn at red lights there. LLMs appear to be excellent means for this and they will have a significant role in the future of robotics and self-driving. We are sharing a technical blog post (https://lnkd.in/enfjXkVy) on LAMBDA, our Multimodal LLM integrated with our autonomy stack, that runs on our in-vehicle hardware platform. We haven’t given direct control of the car to LAMBDA yet, but it already demonstrates advanced understanding and reasoning in various road situations.

Read our blogpost about how to leverage LLMs in self driving!

We are often asked the (good :)) question of how the latest Gen AI advances will impact autonomous driving. Here's a quick sneak peak. From the start of Nuro we have invested in leveraging and building off of the latest and greatest AI advancements. And today our on-road behavior is almost exclusively AI driven -- with "almost" being very important, based on a robust parallel real time validation system we will geek out over in subsequent posts. We're now expanding beyond integrating the fundamental AI breakthroughs behind language models (which we have done for years) towards integrating language models themselves into the stack. It is so cool to see where we have collectively gotten as an industry and what we can already do with this technology. And just wait for what's next...

Revolutionizing Autonomous Driving with SPformer: AI-Driven Multi-Vehicle Collaboration 🚗🤖 How does AI enable vehicles to collaborate seamlessly on the road? In this presentation, I explore SPformer, a cutting-edge Deep Reinforcement Learning (DRL)-based framework that enhances multi-vehicle cooperative decision-making. Want to dive deeper into AI innovations for autonomous vehicles? Check out the presentation for a closer look at how SPformer outperforms traditional models like CNN and GNN! ➡️ 48/100 #AI #AutonomousDriving #ReinforcementLearning #Transformers #VehicleCollaboration #DeepLearning #GNN #100DayChallenge #AIInTransportation #SPformer #AIInnovation

🚗🌱 Charting the Future of Robotics in Autonomous Driving: Japanese Engineers Make a Breakthrough with Musashi 🚗🌱 Japanese engineers have taught the humanoid robot Musashi, featuring a musculoskeletal body, to drive a single-seat car. Musashi managed to steer, operate pedals, accelerate, and brake, responding to traffic signals and car honks. This innovative research, led by Masayuki Inaba from the University of Tokyo, uses artificial muscles for smoother, more human-like movements, enhancing the robot's flexibility and adaptability. Unlike traditional autonomous vehicles with advanced sensors and processors, Musashi's humanoid approach uses conventional car controls (steering wheel and pedals). This method offers versatility, as humanoid robots can perform various tasks beyond driving, such as carrying objects, household chores, and assisting in disaster responses. Musashi's skeleton has lightweight "bones" and 39 joints. It uses 74 artificial muscles driven by electric motors, providing natural movements. The robot's hands and feet have force sensors, and its head features high-resolution cameras for a wide field of view. Algorithms control its actions, ensuring smooth steering and precise pedal operation. Experiments with a Toyota COMS microelectric vehicle showed Musashi successfully steering, maintaining speed, and braking for obstacles. Plans include enhancing the robot's hardware and teaching it to enter and navigate vehicles autonomously. #AutonomousDriving #HumanoidRobots #AI #Innovation #FutureOfDriving #MusashiRobot #Robotics #Technology #SmartCities #MasayukiInaba #UniversityOfTokyo #TransparenTerra https://lnkd.in/efzzAhrv

PARTNERSHIP: dSPACE’s Intempora and BrightDrive collaborate on real-time system development and #data annotation for AVs This partnership integrates #BrightAnnotate's annotation algorithms into Intempora's #RTMaps (Real-Time Multisensor applications) middleware, to improve real-time system development and data annotation for #AVs. This integration enables developers to use the annotation engine through drag-and-drop toolboxes, and enhances RTMaps's data annotation capabilities and accelerates the development of advanced perception #algorithms for #autonomousdriving. "BrightAnnotate is one of the most advanced semi-automated annotation engines. It enables #OEMs and Tier 1 suppliers to develop perception applications for #ADAS and autonomous driving to scale their #machinelearning activities related to training and validation to levels they could not achieve with manual annotation," said Hossam Yahia, CEO of BrightDrive. "The integration of BrightDrive technologies within Intempora software solutions marks an exciting chapter for the automotive industry, added Nicolas du Lac, CEO/CTO at Intempora. Read more here: https://lnkd.in/dPvtxtfw #AutonomousVehicles #AV #SelfDriving #AutonomousDriving #Automotive #AutomotiveIndustry #Transportation #Technology #FutureMobility #Sustainability #AVTExpoStuttgart #AVTExpoCA

Have you tried our elearning module on Quality control in industrial production? One of the most widely used artificial intelligence applications is image recognition using machine vision. Applications of this type are now commonly used in mobile phones, while sensor and control systems in autonomous vehicles are slightly more complex and reliable. This type of artificial intelligence can be deployed immediately in your business and just as quickly transform into tangible economic benefits. Check it out here: https://lnkd.in/dVyqe9r3 #PAITool #AIforSMEs

The latest Nuro technical blog post "LAMBDA: The Nuro Driver's Real-Time Language Reasoning Model" https://lnkd.in/dYBFA_cE shows our latest Research initiatives converted into a practical application. Can we use the groundbreaking results in LLMs (actually, MLLMs, VLMs) in Autonomous Driving? Would it help with challenging situations ("long-tails", especially important for safe L4 driving) where rigid reasoning logic has its limitations? Is it possible to add a human-car interface for a natural way of communication about its decision-making boosting interpretability and explainability? It seems that all these questions have a positive answer, and moreover - we can do it in real-time using the hardware inside our autonomous vehicle platforms! Please check our technical blog post that can provide more insights about architecture, data used, details on state and action reasoning, and even some implementation notes. And stay tuned - a lot of challenges are still waiting for a proper solution, so there is definitely room for future Research, so join us in this endeavor! Thanks to the authors - Aryan Arbabi, Yu (Brian) Yao, Haohuan Wang, Wuming Zhang, Georgie Mathews, and Matt Hanczor (and a long list of people who provided help and support)! #nuro #llm #vlm #mllm #longtail #selfdriving #autonomousdriving