AI-Gateway

<!-- markdownlint-disable MD033 --> # 🧪 [AI Gateway](https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities) labs [](https://github.com/firstcontributions/open-source-badges) ## What's new ✨ ➕ [**AI Gateway workshop**](https://aka.ms/ai-gateway/workshop) provides a comprehensive learning experience using the Azure Portal <div> <a href="https://aka.ms/ai-gateway/workshop" target="_blank"><img src="./images/workshop.png" alt="workshop" width="300"></a> </div> ➕ Refactor most of the labs to use the new [**LLM built-in logging**](https://azure.microsoft.com/en-us/updates?id=491970) that supports streaming completions. ➕ **Realtime API (Audio and Text) with Azure OpenAI 🔥** experiments with the [**AOAI Realtime**](labs/realtime-audio/realtime-audio.ipynb) ➕ **Realtime API (Audio and Text) with Azure OpenAI + MCP tools 🔥** experiments with the [**AOAI Realtime + MCP**](labs/realtime-mcp-agents/realtime-mcp-agents.ipynb) ➕ **Model Context Protocol (MCP) ⚙️** experiments with the [**client authorization flow**](labs/mcp-client-authorization/mcp-client-authorization.ipynb) ➕ the [**FinOps Framework**](labs/finops-framework/finops-framework.ipynb) lab to manage AI budgets effectively 💰 ➕ **Agentic ✨** experiments with [**Model Context Protocol (MCP)**](labs/model-context-protocol/model-context-protocol.ipynb). ➕ **Agentic ✨** experiments with [**OpenAI Agents SDK**](labs/openai-agents/openai-agents.ipynb). ➕ **Agentic ✨** experiments with [**AI Agent Service**](labs/ai-agent-service/ai-agent-service.ipynb) from [Azure AI Foundry](https://azure.microsoft.com/en-us/products/ai-foundry). ## Contents 1. [🧠 AI Gateway](#-ai-gateway) 1. [🧪 Labs with AI Agents](#-labs-with-ai-agents) 1. [🧪 Labs with the Inference API](#-labs-with-the-inference-api) 1. [🧪 Labs based on Azure OpenAI](#-labs-based-on-azure-openai) 1. [🚀 Getting started](#-getting-started) 1. [🔨 Supporting tools](#-supporting-tools) 1. [🏛️ Well-Architected Framework](#-well-architected-framework) <!-- markdownlint-disable-line MD051 --> 1. [🥇 Other Resources](#-other-resources) The rapid pace of AI advances demands experimentation-driven approaches for organizations to remain at the forefront of the industry. With AI steadily becoming a game-changer for an array of sectors, maintaining a fast-paced innovation trajectory is crucial for businesses aiming to leverage its full potential. **AI services** are predominantly accessed via **APIs**, underscoring the essential need for a robust and efficient API management strategy. This strategy is instrumental for maintaining control and governance over the consumption of **AI models**, **data** and **tools**. With the expanding horizons of **AI services** and their seamless integration with **APIs**, there is a considerable demand for a comprehensive **AI Gateway** pattern, which broadens the core principles of API management. Aiming to accelerate the experimentation of advanced use cases and pave the road for further innovation in this rapidly evolving field. The well-architected principles of the **AI Gateway** provides a framework for the confident deployment of **Intelligent Apps** into production. ## 🧠 AI Gateway  This repo explores the **AI Gateway** pattern through a series of experimental labs. The [AI Gateway capabilities](https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities) of [Azure API Management](https://learn.microsoft.com/azure/api-management/api-management-key-concepts) plays a crucial role within these labs, handling AI services APIs, with security, reliability, performance, overall operational efficiency and cost controls. The primary focus is on [Azure AI Foundry models](https://learn.microsoft.com/en-us/azure/ai-foundry/what-is-azure-ai-foundry), which sets the standard reference for Large Language Models (LLM). However, the same principles and design patterns could potentially be applied to any third party model. Acknowledging the rising dominance of Python, particularly in the realm of AI, along with the powerful experimental capabilities of Jupyter notebooks, the following labs are structured around Jupyter notebooks, with step-by-step instructions with Python scripts, [Bicep](https://learn.microsoft.com/azure/azure-resource-manager/bicep/overview?tabs=bicep) files and [Azure API Management policies](https://learn.microsoft.com/azure/api-management/api-management-howto-policies): ## 🧪 Labs with AI Agents <!-- MCP Client Authorization --> ### [**🧪 MCP Client Authorization**](labs/mcp-client-authorization/mcp-client-authorization.ipynb) Playground to experiment the [Model Context Protocol](https://modelcontextprotocol.io/) with the [client authorization flow](https://modelcontextprotocol.io/specification/2025-03-26/basic/authorization