LiteLLM 1.108.12 (LTS) Fixes Prompt Caching, API Edge Cases, and Guardrail Issues
BerriAI releases LiteLLM version 1.108.12 (LTS) with six targeted fixes to prompt caching, API errors, streaming behavior, and guardrail enforcement on chat, message, and batch endpoints.
What changed?
LiteLLM 1.108.12 (LTS) delivers six bug fixes across chat, message, Vertex batch, and model_router APIs. Key changes include fixing GPT-6.2 prompt caching when system prompts are sent to /v1/chat/completions or /v1/messages with store: false, resolving failure cases for Foundry-based Azure model routing, addressing every-other-row failures for json_schema in Vertex batch files, correcting improper responses involving tool names over 64 characters, stopping OpenAI models from streaming after the caller closes the connection, and hardening tool_policy guardrails for long tool names.
Why does it matter to an everyday developer?
These fixes improve reliability and correctness in chat and batch AI workflows. Developers using GPT-6.2 via LiteLLM will get more predictable prompt caching, particularly in workflows that bypass result storage. Azure users on the Foundry API gain more consistent routing and response handling. Developers using Vertex AI batch schema or tools with long names will see fewer unexplained errors. Applications relying on streaming from OpenAI models will no longer process upstream data after disconnection, reducing wasted resources. Guardrails now more reliably block tool usage based on policy, even for unusually long tool names.
What can the developer do now?
Update to LiteLLM 1.108.12 LTS to gain these fixes. Pull the lts Docker image tag to stay current with LTS patches. There are no breaking changes or action-required migrations for this release, so upgrades are expected to be seamless. If you encountered any of the fixed issues previously—particularly around prompt caching, API errors, or long tool names—test your workflows after updating to confirm resolution.

