Multimodal input

Send images, audio, video and documents alongside your text. Content becomes an array of parts instead of a plain string.

Which model takes what#

Images#

By URL, or inline as a data URI if the file is local.

1response = client.chat.completions.create(
2    model="upfyn-yuva",
3    messages=[{
4        "role": "user",
5        "content": [
6            {"type": "text", "text": "What is the total on this invoice?"},
7            {"type": "image_url",
8             "image_url": {"url": "https://example.com/invoice.png"}},
9        ],
10    }],
11)

Getting good answers from images

• Ask a specific question. “What is the total?” beats “describe this” every time.
• Send the image at a readable size — downscaling a dense screenshot until the text blurs is the most common cause of a wrong answer.
• Several images in one message is fine. Say which is which in the text part.
• Base64 counts toward your request size. For anything you will reference more than once, upload it instead.

Documents and reuse#

Upload once, reference by id. This is the right approach for PDFs, and for any file used across several turns — you pay to send the bytes once.

1# 1. Upload once
2uploaded = client.files.create(file=open("contract.pdf", "rb"), purpose="assistants")
3 
4# 2. Reference it by id — no re-upload on later turns
5response = client.chat.completions.create(
6    model="upfyn-yuva",
7    messages=[{
8        "role": "user",
9        "content": [
10            {"type": "text", "text": "Summarise the termination clause."},
11            {"type": "file", "file": {"file_id": uploaded.id}},
12        ],
13    }],
14)

• Uploads cap at 100 MB; larger returns 413 file_too_large.
• Files are private to your account. A file id alone fetches nothing — every read matches the owner.
• Manage them from Files or GET /v1/files.

Audio and video#

Audio parts go to Yuva and Rishi; video is Rishi only. Both work the same way as images — a content part with a URL or a file_id.

Producing media, not reading it#

This page is about input. To make an image, speech or a video, see Generating media.