lemonade

## 🍋 Lemonade: Local LLM Serving with GPU and NPU acceleration <p align="center"> <a href="https://discord.gg/5xXzkMu8Zk"> <img src="https://img.shields.io/badge/Discord-7289DA?logo=discord&logoColor=white" alt="Discord" /></a> <a href="https://github.com/lemonade-sdk/lemonade/tree/main/test" title="Check out our tests"> <img src="https://github.com/lemonade-sdk/lemonade/actions/workflows/test_lemonade.yml/badge.svg" alt="Lemonade tests" /></a> <a href="docs/README.md#installation" title="Check out our instructions"> <img src="https://img.shields.io/badge/Windows-11-0078D6?logo=windows&logoColor=white" alt="Windows 11" /></a> <a href="https://lemonade-server.ai/#linux" title="Ubuntu 24.04 & 25.04 Supported"> <img src="https://img.shields.io/badge/Ubuntu-24.04%20%7C%2025.04-E95420?logo=ubuntu&logoColor=white" alt="Ubuntu 24.04 | 25.04" /></a> <a href="docs/README.md#installation" title="Check out our instructions"> <img src="https://img.shields.io/badge/Python-3.10--3.13-blue?logo=python&logoColor=white" alt="Made with Python" /></a> <a href="https://github.com/lemonade-sdk/lemonade/blob/main/docs/contribute.md" title="Contribution Guide"> <img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome" /></a> <a href="https://github.com/lemonade-sdk/lemonade/releases/latest" title="Download the latest release"> <img src="https://img.shields.io/github/v/release/lemonade-sdk/lemonade?include_prereleases" alt="Latest Release" /></a> <a href="https://tooomm.github.io/github-release-stats/?username=lemonade-sdk&repository=lemonade"> <img src="https://img.shields.io/github/downloads/lemonade-sdk/lemonade/total.svg" alt="GitHub downloads" /></a> <a href="https://github.com/lemonade-sdk/lemonade/issues"> <img src="https://img.shields.io/github/issues/lemonade-sdk/lemonade" alt="GitHub issues" /></a> <a href="https://github.com/lemonade-sdk/lemonade/blob/main/LICENSE"> <img src="https://img.shields.io/badge/License-Apache-yellow.svg" alt="License: Apache" /></a> <a href="https://github.com/psf/black"> <img src="https://img.shields.io/badge/code%20style-black-000000.svg" alt="Code style: black" /></a> <a href="https://star-history.com/#lemonade-sdk/lemonade"> <img src="https://img.shields.io/badge/Star%20History-View-brightgreen" alt="Star History Chart" /></a> </p> <p align="center"> <img src="https://github.com/lemonade-sdk/assets/blob/main/docs/banner_02.png?raw=true" alt="Lemonade Banner" /> </p> <h3 align="center"> <a href="https://lemonade-server.ai">Download</a> | <a href="https://lemonade-server.ai/docs/">Documentation</a> | <a href="https://discord.gg/5xXzkMu8Zk">Discord</a> </h3> Lemonade helps users run local LLMs with the highest performance by configuring state-of-the-art inference engines for their NPUs and GPUs. Startups such as [Styrk AI](https://styrk.ai/styrk-ai-and-amd-guardrails-for-your-on-device-ai-revolution/), research teams like [Hazy Research at Stanford](https://www.amd.com/en/developer/resources/technical-articles/2025/minions--on-device-and-cloud-language-model-collaboration-on-ryz.html), and large companies like [AMD](https://www.amd.com/en/developer/resources/technical-articles/unlocking-a-wave-of-llm-apps-on-ryzen-ai-through-lemonade-server.html) use Lemonade to run LLMs. ## Getting Started 1. **Install**: [Windows](https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) · [Ubuntu](https://lemonade-server.ai/install_options.html) · [Source](https://lemonade-server.ai/install_options.html) 2. **Get Models**: Browse and download with the [Model Manager](#model-library) 3. **Chat**: Try models with the built-in chat interface 4. **Connect**: Use Lemonade with your favorite apps: <p align="center"> <a href="https://lemonade-server.ai/docs/server/apps/open-webui/" title="Open WebUI"><img src="https://raw.githubusercontent.com/lemonade-sdk/assets/refs/heads/main/partner_logos/openwebui.jpg" alt="Open WebUI" width="60" /></a>&nbsp;&nbsp;<a href="https://n8n.io/integrations/lemonade-model/" title="n8n"><img src="https://raw.githubusercontent.com/lemonade-sdk/assets/main/app/marketplace/n8n.png" alt="n8n" width="60" /></a>&nbsp;&nbsp;<a href="https://github.com/amd/gaia" title="Gaia"><img src="https://raw.githubusercontent.com/lemonade-sdk/assets/main/app/marketplace/gaia.png" alt="Gaia" width="60" /></a>&nbsp;&nbsp;<a href="https://github.com/lemonade-sdk/infinity-arcade" title="Infinity Arcade"><img src="https://raw.githubusercontent.com/lemonade-sdk/assets/main/app/marketplace/infinity_arcade.png" alt="Infinity Arcade" width="60" /></a>&nbsp;&nbsp;<a href="https://lemonade-server.ai/docs/server/apps/continue/" title="Continue"><img src="https://raw.githubusercontent.com/lemonade-sdk/assets/main/app/marketplace/continue.png" alt="Continue" width="60" /></a>&nbsp;&nbsp;<a href="https://marketplace.visualstudio.com/items?itemName=lemonade-sdk.lemonade-sdk" titl