UpfynUpfyn
Upfyn Web

Client SDKs

Python

The official openai package, pointed at Upfyn. Everything here runs as written.

Setting up the client#

client.py
import os
from openai import OpenAI
 
client = OpenAI(
    base_url="https://ai.upfyn.com/v1",
    api_key=os.environ["UPFYN_API_KEY"],
    timeout=60.0,        # generous: a reasoning turn can take a while
    max_retries=0,       # retry yourself, so you control which codes are retried
)

Turn the SDK's own retries off

max_retries=0 looks backwards but it is the right default here: the SDK will happily retry a 402, which can never succeed. Handle retries yourself so you only retry 429 and 5xx — see Errors.

A basic call#

One turn
response = client.chat.completions.create(
    model="upfyn-yuva",
    messages=[
        {"role": "system", "content": "You answer in one short paragraph."},
        {"role": "user", "content": "Why is my invoice higher this month?"},
    ],
    max_tokens=400,
)
 
print(response.choices[0].message.content)
print("cost:", response.model_extra.get("credits_used"))

credits_used is an Upfyn field, so the typed model hides it — model_extra is where the SDK keeps everything it did not expect.

Streaming#

Collecting as you print
stream = client.chat.completions.create(
    model="upfyn-yuva",
    messages=[{"role": "user", "content": "Draft a status update."}],
    stream=True,
    stream_options={"include_usage": True},
)
 
pieces = []
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        piece = chunk.choices[0].delta.content
        pieces.append(piece)
        print(piece, end="", flush=True)
 
text = "".join(pieces)
  • Guard on chunk.choices — the final usage chunk has an empty list.
  • Accumulate into a list and join once; repeated string concatenation in a hot loop is the slow way.

Concurrency#

AsyncOpenAI with a semaphore
import asyncio
from openai import AsyncOpenAI
 
client = AsyncOpenAI(base_url="https://ai.upfyn.com/v1", api_key=KEY)
gate = asyncio.Semaphore(8)          # stay under the per-minute limit
 
async def ask(question: str) -> str:
    async with gate:
        response = await client.chat.completions.create(
            model="upfyn-bala",
            messages=[{"role": "user", "content": question}],
        )
        return response.choices[0].message.content
 
async def main():
    answers = await asyncio.gather(*(ask(q) for q in questions))

Your rate limit is the real ceiling

A bare asyncio.gather over a hundred prompts hits 60 requests per minute immediately and starts collecting 429s. Bound the fan-out.

Errors#

Distinguishing what to retry
from openai import APIStatusError, APIConnectionError
 
try:
    response = client.chat.completions.create(...)
except APIStatusError as err:
    if err.status_code == 402:
        raise OutOfCredits(err.response.json()["error"]["message"])
    if err.status_code == 429:
        wait = float(err.response.headers.get("Retry-After") or 5)
        ...
    raise
except APIConnectionError:
    ...   # network, not the API — safe to retry

Upfyn-only parameters

extra_body
client.chat.completions.create(
    model="upfyn-yuva",
    messages=messages,
    extra_body={
        "persist_session": True,
        "chat_ref": "ticket-8821",
        "upfyn_tag": "support",
    },
)
PreviousOverviewNext TypeScript