Source: Arm Author: Arm
     Arm Holdings Limited (NASDAQ: ARM, hereinafter referred to as "Arm") today announced a new partnership with Alibaba Taotian Group's lightweight deep learning framework MNN. Through the integration of Arm KleidiAI, the two parties successfully put multimodal artificial intelligence (AI) workloads on mobile devices equipped with Arm cpus using Alibaba's instruction-adjusted Qwen2-VL-2B-Instruct model. This version of the Tongyi model is designed for image understanding from end-side devices, text-to-image reasoning, and multimodal generation across multiple languages. This collaboration significantly improves the performance of end-to-side multimodal AI workloads and brings a new user experience.

    Stefan Rosinger, Senior Director of Product Management at Arm's Devices Division, said: "We are in the midst of the AI revolution and are witnessing the rise of multimodal AI models. These models can process and understand multiple data types, including text, images, audio, video, and sensor data. However, due to the power constraints and memory constraints of the hardware itself, as well as the complexity of processing multiple data types at the same time, deploying these advanced multimodal models on end-side devices is a challenge."
    Arm Kleidi is the ideal solution to these challenges, providing seamless performance optimization for all AI inference workloads running on Arm cpus. KleidiAI is a set of lightweight and high-performance open source Arm routines designed for AI acceleration that are currently integrated into the latest versions of major end-to-end AI frameworks, including ExecuTorch, Llama.cpp, LiteRT (via XNNPACK), and MediaPipe. It enables millions of developers to automatically reap significant improvements in AI performance without additional action.

Accelerate response time for end-to-side multimodal AI use cases
    By integrating KleidiAI with MNN, the Arm and MNN teams measured the acceleration performance of the Qwen2-VL-2B-Instruct model, showing improved running and response speed in critical AI multimodal application scenarios on the end side. This upgrade will bring a better user experience to Alibaba's many customer-centric apps.
    These use case response improvements were driven by a 57% improvement in model pre-filling (when the AI model processes the prompt input before generating a response) and a 28% improvement in decoding (when the AI model generates text from the AI model after processing the prompt). In addition, KleidiAI integration can further facilitate efficient processing of AI workloads on end-to-end devices by reducing the overall computing cost of multimodal workloads. Millions of developers running applications and workloads using popular AI frameworks, including MNN frameworks, can enjoy these performance and efficiency gains in applications and workloads for edge-side devices.
    Xu Dong, general manager of Aliyun Tongyi large model business, said: "We are very pleased to see the deep technical cooperation between the Big Model and Arm KleidiAI and the MNN team, and through the integrated accelerated optimization of the MNN end-to-end inference framework and Arm KleidiAI, we have successfully achieved a significant reduction in the inference delay and a significant increase in the energy efficiency ratio of the big model." This groundbreaking collaboration not only fully validates the practical potential of large models in mobile terminals, but also enables users to experience the inclusive value of next-generation AI at their fingertips. We look forward to the three parties continuing to work together to break through the boundaries of computing power with technological innovation and jointly open a new chapter in end-to-end intelligence."
    Jiang Xiaotang, head of MNN business technology of Alibaba Taotan Group, said: "The MNN reasoning framework and Arm KleidiAI do deep technology integration, which has made a new breakthrough in the acceleration of the end-side large model. Through our joint optimization of the underlying architecture, the end-to-end reasoning efficiency of Tongyi's large model has been greatly improved, successfully crossing the technical gap between limited computing power and complex AI capabilities. This achievement is not only the crystallization of the technical breakthrough of the MNN team, but also a vivid interpretation of the cross-border collaboration power. We look forward to continuing to work together to deepen the end-to-end computing ecosystem in the future, so that every mobile terminal can 

KleidiAI integration demonstration at MWC
    At this year's Mobile World Congress (MWC), Arm presented the results of this collaboration in the event booth (Hall 2, Booth I60), where the demonstration highlighted how the model understands multiple combinations of visual and text input and distills the content in the image. The demonstration was performed on smartphones equipped with the MediaTek Tianguet 9400 mobile system-on-chip (SoC), including the vivo X200 series.

Make the leap to multi-modal AI experience
    KleidiAI's integration with the MNN framework supported by the Alibaba Tongyi Qiwen model has successfully brought significant user experience improvements to multimodal AI workloads running on the Arm CPU side. These great experiences are now available on mobile devices, and many customer-facing applications are already benefiting from KleidiAI's benefits. Going forward, KleidiAI's seamless optimization for AI workloads will continue to empower developers to deliver more complex multimodal experiences on end-side devices. This will set the stage for the next wave of intelligent computing and take an exciting step forward in the continued evolution of AI.

About Arm
    As the industry's most powerful and energy-efficient computing platform, Arm covers 100% of the world's connected population with unparalleled scale. Arm delivers advanced solutions to meet the never-ending demand for computing, enabling the world's leading technology companies to unlock unprecedented AI experiences and performance. Arm is working with the world's most extensive computing ecosystem and 20 million software developers to build the future of AI on the Arm platform.