Lampi & Ampere Collaboration at CloudFest 2024: Pioneering Sustainable AI Solutions

Efficient and sustainable cloud computing solutions has become a hot topic in the deployment of AI solutions, in particular regarding the use of the two most used processing systems: CPU (Central Processing Unit)  and GPU (Graphics Processing Unit).

The use of CPUs over GPUs is a complex and technical question. As AI model training involves performing complex mathematical on large datasets requiring a significant amount of processing power, training an AI model using GPUs is generally much faster and more efficient. However, CPUs can today deliver robust performance, serving as cost-effective and more sustainable options for running AI-based solutions during inference stage compared to GPUs.

At Lampi, we are deeply committed to efficiency and sustainability and have adopted a strategic approach to mitigate environmental AI impact without compromising on performance, which includes a hybrid approach prioritizing the use of CPUs over GPUs wherever feasible.

After benchmarking AI CPUs on the market, it appears that Ampere Computing COPARM stands at the forefront of revolutionizing the cloud infrastructure landscape by pioneering innovative approaches by integrating advanced semiconductor technologies and optimizing the chip's architecture.

Our work and expertise have been recognized in our collaboration with Ampere Computing at the CloudFest 2024, the world's largest cloud industry event, highlighting a shared commitment to advancing AI technology while prioritizing environmental consciousness.

Our collaboration showcases how Lampi, leveraging Ampere's Cloud Native Processors, redefines efficiency in AI applications in the context of a real use-case for enterprises.

The collaboration took stage at CloudFest with a compelling demonstration of Lampi's software capabilities, empowered with our fine-tuned versions of open-source generative AI models (including Whisper, originally developed by OpenAI and used especially for automatic speech recognition), all running on an Ampere-based cloud instance.

Our live demo shows a real-time voice-to-text and voice-to-voice chatbot interaction, incoming real-time input from a dedicated external microphone available to the visitors of the Ampere Cloudfest booth. The demo runs at a real-time performance level with latency low enough and a per second token generation rate high enough to satisfy the user needs. The voice-to-voice chatbot interaction is currently not presented at the event due to technical limitations posed by the show floor environment.

The demo serves not just as a technical showcase, but also as an educational experience for attendees, enlightening them on the potential of sustainable AI technology.

CPUs have their places in specific inference stage, while GPUs will still be at the heart of training, especially with the larger models, and other AI use cases, our collaboration showcases the following key benefits:

• Low latency real-time interaction for the use case defined: The collaboration meets the critical low latency requirements essential for real-time generative AI chatbot interaction;
• Scalability through optimized architecture;
• Optimal price-performance ratio: our solution illustrates the best price-performance in GPU-Free AI inference, highlighting cost efficiency in both cloud and edge deployment scenarios.

As we stand on the brink of a new era in AI and computing, this collaboration serves as a blueprint for the industry, demonstrating that it is possible to achieve technological excellence without compromising on environmental integrity.

For more information, you can visit:

• https://amperecomputing.com/events/cloudfest
• https://amperecomputing.com/assets/Lampi_Cloudfest_Demo_Brochure_6b5a6e219a.pdf

At Lampi, our mission is to accelerate the development and deployment of AI applications and automatization of workflows for enterprises. That’s why we are excited to share our knowledge and best practices for organizations to deploy AI for real business impact. 

Discover more about our AI solutions on our website

Don't forget to follow us on LinkedIn, Twitter, and Instagram!