Penguin Solutions Boosts AI Inference with OriginAI

Penguin Solutions, Inc. has announced an expansion of its OriginAI® portfolio, introducing new solutions designed to address growing challenges in AI inference, particularly around GPU memory limitations, latency, and scalability. The enhanced offerings aim to help enterprises run large-scale AI workloads more efficiently while improving performance consistency and deployment speed.

The new OriginAI solutions are built to complement NVIDIA RTX PRO 6000 and NVIDIA B300 GPU architectures by integrating large memory appliances. This approach enables organizations to handle larger context sizes, improve concurrency, and reduce latency critical factors for real-time AI applications. By focusing on memory optimization alongside compute power, Penguin Solutions aims to overcome common bottlenecks that impact inference performance.

Drawing on more than 30 years of experience in advanced memory solutions and over 3.3 billion hours of GPU runtime expertise, the company has designed OriginAI to deliver production-ready inference capabilities. The platform emphasizes predictable performance, improved GPU utilization, and enhanced infrastructure reliability, helping organizations accelerate time to value for AI deployments.

A key component of the expanded portfolio is the MemoryAI™ KV Cache Server, a CXL-based solution that increases key-value cache capacity. This enables support for longer context windows and high-concurrency workloads while maintaining low latency. The server is compatible with NVIDIA’s Dynamo framework and is designed to provide a cost-efficient foundation for next-generation AI deployments.

In addition, OriginAI includes ICE ClusterWare™ software, an intelligent management layer that transforms hardware into a fully optimized AI cluster. The software provides real-time health monitoring, automated issue resolution, and workload isolation, ensuring secure and reliable performance in multi-tenant environments. The OriginAI portfolio offers multiple configurations tailored to different enterprise use cases. Systems based on NVIDIA RTX PRO 6000 GPUs are suited for applications such as enterprise copilots, retrieval-augmented generation (RAG), code assistance, and document summarization. These configurations provide cost-effective and energy-efficient performance for mid-sized AI models.

For more demanding workloads, NVIDIA B300-based architectures deliver higher memory bandwidth and scalability. These are designed for enterprise-wide AI platforms, long-context assistants, and agentic AI applications that require large-scale processing and future-ready infrastructure.

Penguin Solutions’ expanded platform is designed to support a wide range of industries where low-latency inference is critical. In financial services, it enables real-time fraud detection and high-frequency trading. In healthcare, it supports time-sensitive applications such as diagnostics, patient monitoring, and medical translation. In retail, it enhances personalization, inventory optimization, and real-time decision-making. By combining advanced memory architecture, GPU integration, and intelligent cluster management, Penguin Solutions is positioning OriginAI as a comprehensive platform for scalable, high-performance AI inference. The expansion reflects a broader industry shift toward optimizing infrastructure not just for training models, but for delivering real-time, production-grade AI outcomes.