Transformers v5.20.0 Adds Muse Spark Open and Unified Exporter Quantization

What changed?

Transformers v5.20.0 introduces support for multiple new models, most notably Meta’s Muse Spark Open, a large multimodal mixture-of-experts model that processes text, image, audio, and video inputs. Additional new model integrations include Ling 3.0 Flash, Kolibri 1, MiniCPM-V 4.7, UnlimitedOcr, UltraBERT, and MOSS-Transcribe-Diarize. This release also debuts a unified quantization API for model exporters supporting multiple backends (Dynamo, ONNX, OpenVINO, ExecuTorch) and integrates quantizers across PT2E, ONNX Runtime, and NNCF. Breaking changes have been made to the cache API, the handling of entity tokens in LUKE/mLUKE tokenizers, and kernel requirements.

Why does it matter to an everyday developer?

The new Muse Spark Open model allows developers to build and experiment with agentic coding systems that use multimodal instructions, multi-file editing, and debugging across large context windows (up to 1,048,576 tokens). The integrated unified quantization API simplifies optimizing models for inference on various hardware, reducing model size and improving performance without requiring manual backend-specific adjustments. The release covers both language and vision models, with expanded options for long-context, multimodal, and specialized tasks like transcription, OCR, or document intelligence. Breaking changes require code that previously relied on deprecated cache methods or older tokenization patterns to update accordingly, potentially impacting ongoing projects.

What can the developer do now?

• Leverage Muse Spark Open and other new models for agentic, multimodal, or long-context applications.
• Utilize the unified quantization API during model export to optimize custom or third-party models for deployment on various inference backends.