Structured Outputs

Return structured data from your models.

OneRouter supports structured outputs for compatible models, ensuring responses follow a specific JSON Schema format. This feature is particularly useful when you need consistent, well-formatted responses that can be reliably parsed by your application.

Overview

Structured outputs allow you to:

• Enforce specific JSON Schema validation on model responses

• Get consistent, type-safe outputs

• Avoid parsing errors and hallucinated fields

• Simplify response handling in your application

Using Structured Outputs

To use structured outputs, include a response_format parameter in your request, with type set to json_schema and the json_schema object containing your schema:

Python

import requests
import json

response = requests.post(
  url="https://llm.onerouter.pro/v1/chat/completions",
  headers={
    "Authorization": "Bearer <API KEY>",
    "Content-Type": "application/json"
  },
  data=json.dumps({
    "model": "{{MODEL}}",
    "messages": [
      {"role": "user", "content": "What's the weather like in London?"}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "weather",
        "strict": True,
        "schema": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City or location name"
            },
            "temperature": {
              "type": "number",
              "description": "Temperature in Celsius"
            },
            "conditions": {
              "type": "string",
              "description": "Weather conditions description"
            }
          },
          "required": ["location", "temperature", "conditions"],
          "additionalProperties": False
        }
      }
    }
  })
)
print(response.json()["choices"][0]["message"]["content"])

The model will respond with a JSON object that strictly follows your schema:

{
	"conditions": "Overcast with light rain showers",
	"location": "London, United Kingdom",
	"temperature": 12.5
}

Model Support

Structured outputs are supported by select models.

You can find a list of models that support structured outputs on the models page.

• OpenAI models (GPT-4o and later versions) Docs

• All Fireworks provided models Docs

To ensure your chosen model supports structured outputs:

1. Check the model's supported parameters on the models page

2. Include response_format and set type: json_schema in the required parameters

Best Practices

1. Include descriptions: Add clear descriptions to your schema properties to guide the model

2. Use strict mode: Always set strict: true to ensure the model follows your schema exactly

Streaming with Structured Outputs

Structured outputs are also supported with streaming responses. The model will stream valid partial JSON that, when complete, forms a valid response matching your schema.

To enable streaming with structured outputs, simply add stream: true to your request:

{
  "stream": true,
  "response_format": {
    "type": "json_schema",
    // ... rest of your schema
  }
}

Error Handling

When using structured outputs, you may encounter these scenarios:

1. Model doesn't support structured outputs: The request will fail with an error indicating lack of support

2. Invalid schema: The model will return an error if your JSON Schema is invalid