LiteLLM v1.111.0-rc.3 Delivers Critical Fixes for Stream Reliability and Cost Tracking
LiteLLM v1.111.0-rc.3 addresses API stream reliability, cost tracking, prompt caching, authentication, and deployment issues, improving stability for developers integrating LLMs from Anthropic, Vertex AI, Azure AI, and OpenAI.
What changed?
LiteLLM v1.111.0-rc.3 introduces a series of bug fixes focused on stream reliability, cost tracking, authentication, caching, and deployment. Notable changes include correcting 400 errors on Anthropic/Claude models when using 'temperature=1', fixing duplicate or extra-billed events and fallback behaviors in streamed responses, and ensuring system prompts are cached for GPT-6.2 on relevant endpoints. The release also resolves issues with cost tracking for Anthropic API keys, organizational spend tracking, JWT key mapping latency, and improves memory management and Postgres connection usage for proxy deployments.
Why does it matter to an everyday developer?
Developers integrating LiteLLM as an API layer for LLM inferencing benefit from improved request reliability, accurate billing, and reduced operational overhead. The fixes address critical cases such as failed or duplicate streamed responses that could result in extra costs or inconsistent user experiences. Prompt caching improvements enhance performance for GPT-6.2, while fixes to authentication, cost tracking, and deployment issues provide more secure, maintainable, and resource-efficient deployments, particularly in production and multi-tenant environments.
What can the developer do now?
Developers using Anthropic, Vertex AI, Azure AI, or OpenAI models via LiteLLM should upgrade to v1.111.0-rc.3 to benefit from the bug fixes and stability improvements. No breaking changes are introduced, so upgrades are low risk and require no code changes. For those who manage API keys, cost tracking, or deploy proxies at scale, this release fixes issues that may have led to untracked spend or inefficient database use. Verification of the new images using Cosign is supported per release instructions.
