litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm is an open-source agent framework AI agent maintained by BerriAI. It ranks #39 on The Agentic Leaderboard with an overall score of 84.4 / 100, scoring 97.4% on reliability and 71.2% on tool selection quality. The project has 49,648 GitHub stars.

Add this badge to your README

[![Agentic Leaderboard](https://www.theagenticleaderboard.com/badges/litellm.svg)](https://www.theagenticleaderboard.com/agent/litellm)