@ai-sdk/typesafe-ai 3.0.16 Adds Support for Ordered Parts and Image Inputs in Decision State

What changed?

@ai-sdk/typesafe-ai version 3.0.16 introduces ordered text, file, and JSON parts to its experimental decision state. It now supports image input in OpenAI Decisions and language model adapters. The Gateway and TypeSafe AI retain their support for only string and JSON input formats and will reject file inputs. Public state forms are normalized into arrays of parts before being sent to decision providers. Arrays used as direct state inputs now consist of decision state parts, and JSON arrays should be wrapped in an object or 'json' part. Decision providers receive well-demarcated text and JSON parts. Additionally, file bytes are serialized as base64 for telemetry purposes, and model-call spans record normalized state parts.

Why does it matter to an everyday developer?

Developers can now create more flexible and multimodal decision pipelines by combining ordered text, files, and JSON objects in the decision state for experimental features. Image input support in decision workflows for OpenAI models makes it easier to incorporate visual data. The changes in state normalization mean developers must prepare state inputs more carefully, especially when using arrays or JSON data, to ensure compatibility and clarity. The requirement to wrap JSON arrays and the consistent formatting for parts can prevent bugs or misinterpretations when building complex integrations.

What can the developer do now?

Developers can start passing images as input for OpenAI Decision workflows and use the new ordered part format for more structured and expressive decision states in experimental usage. When supplying arrays as state, developers must ensure they use the expected format by wrapping JSON arrays in objects or using the 'json' part. Direct file uploads to the Gateway or TypeSafe AI remain unsupported, so maintain use of string or JSON formats in production workflows.