Alibaba Cloud has integrated its Large Multimodal Model (LMM) solution with Nvidia’s automotive platform to improve smart mobility experiences for China-based automakers.
The collaboration, which also involves Alibaba’s smart cockpit solutions provider Banma, aims to revolutionize in-car interactions and autonomous driving capabilities.
The collaboration was followed by the introduction of a large multimodal model (LMM) solution by Alibaba Group Holding. cloud computing services unit. The LLM solution is designed for automotive applications that Alibaba Cloud jointly developed with Nvidia and startup Banma Network Technology, Alibaba’s smart cockpit solutions provider.
The partnership also sees Alibaba Cloud’s proprietary Qwen large language model, including the Qwen2-7B and Qwen2-VL vision language model, built into the NVIDIA DRIVE AGX Orin platform for autonomous vehicles. This integration, along with NVIDIA’s model acceleration technology, promises to reduce computational costs and minimize latency in real-time processing of complex tasks.
This marks the first integration of Alibaba Cloud’s large AI models in the The US-based semiconductor powerhouse Drive automotive platform. Major Chinese electric vehicle (EV) manufacturers, including Li Auto, Great Wall Motors, Geely Auto’s Zeekr Intelligent Technology and Xiaomi’s EV unit, use this platform to power their next-generation fleets.
Leveraging Qwen’s advanced capabilities enables in-car voice assistance to engage in dynamic, multi-turn conversations and offer recommendations. These range from providing information about nearby landmarks to suggesting when to turn on the lights in inclement weather conditions.
The LMM solution also includes Alibaba Cloud’s Mobile Agent, a multi-modal AI agent for edge devices. This technology is intended to expand the capabilities of voice assistants in cars, allowing users to execute complex voice commands, such as ordering food through delivery apps, while on the move.
NVIDIA’s participation brings substantial benefits to the partnership. The company’s expertise in artificial intelligence and graphics processing has been instrumental in optimizing the performance of Alibaba Cloud models on automotive-grade hardware. This optimization ensures that AI-powered functions can operate efficiently within the limitations of the vehicles’ computer systems.
Overall, the collaboration between Alibaba Cloud and Nvidia is expected to have implications for the broader automotive industry in China. By making this solution available to China-based automakers, Alibaba Cloud and NVIDIA are potentially accelerating the adoption of AI-powered technologies in vehicles in one of the world’s largest automotive markets.
Looking ahead, Alibaba Cloud plans to expand its collaboration with NVIDIA. The companies intend to adapt the Qwen models for NVIDIA DRIVE Thor, NVIDIA’s next-generation centralized automotive computing platform. This platform combines advanced driver assistance, autonomous driving and cockpit AI capabilities into one secure system.
Separately, Nvidia is also working with Alibaba Cloud on infrastructure “to meet people’s extreme demand for computing power,” according to Nvidia global vice president He Tao in a subforum hosted by the American company at the Apsara Conference.
According to Nvidia, Nvidia and Alibaba Cloud are carrying out a series of initiatives related to LLM for accelerated computing. These initiatives use parallel processing to speed up work and increase energy efficiency in demanding applications. Both companies are also working to create customized solutions to help “traditional firms” move their LLM operations to the cloud.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
