SwiftRelay

← All sprint notes

First Lap: Send an Image to a Vision Endpoint

Protocol pit stopEN

Your first image request needs three things: a model that accepts images, an image the endpoint can read, and a message that tells the model what to inspect. This walkthrough uses SwiftRelay’s OpenAI-compatible interface to send one image and get a useful text response.

1. Prepare your SwiftRelay key and vision model

Sign up for SwiftRelay, create an API key, and keep it in an environment variable named SWIFTRELAY_API_KEY. SwiftRelay is a pay-as-you-go relay with per-token billing, so you can start with a small test rather than a large integration.

Select a model whose current route explicitly supports image input. Gemini 2.5 Flash is a candidate to check in the available model list. Copy the exact API model identifier shown there; a display name is not necessarily the identifier your request must send.

For an OpenAI-compatible SDK, set the base URL to https://api.fastapirelay.com/v1. For the raw HTTP request below, use POST https://api.fastapirelay.com/v1/chat/completions. Avoid appending another /v1 when your configured base URL already includes it.

2. Choose an image the endpoint can read

For the first run, use a clear JPEG or PNG containing one obvious subject. A product label, receipt, or simple chart makes the result easy to verify. Avoid sensitive personal information while testing.

You can supply a remotely accessible HTTPS image URL or, where supported by the selected route, a base64 data URL. A remote URL must return the actual image without requiring a browser login, session cookie, or interactive download page.

  • Remote image: convenient when the file is already hosted and accessible to the service fetching it.
  • Data URL: useful for a local file, but it increases the request size and remains subject to the route’s payload limits.

Check the selected model’s supported formats, image dimensions, and request limits before using a large file. These constraints are not identical across providers.

3. Build a message containing text and image parts

In a chat-completions request, image input belongs inside the user message’s content array. Put the instruction in a text part and the image reference in an image_url part. Do not paste an image URL into a plain text prompt and assume the model will retrieve it.

Use this JSON body, replacing VISION_MODEL_ID with the exact model identifier and IMAGE_HTTPS_URL with your image’s accessible address:

{"model":"VISION_MODEL_ID","messages":[{"role":"user","content":[{"type":"text","text":"Describe the visible object, then transcribe any readable text. If something is unclear, say so instead of guessing."},{"type":"image_url","image_url":{"url":"IMAGE_HTTPS_URL"}}]}]}

This prompt creates two checks: whether the model recognizes the subject and whether it can read visible text. Explicit uncertainty instructions help prevent a blurry label from becoming an invented transcription.

4. Send the request with authentication

Save the JSON body as request.json. Send it with an HTTP client using these headers: Authorization: Bearer YOUR_API_KEY and Content-Type: application/json.

From a shell with your environment variable set, run:

curl https://api.fastapirelay.com/v1/chat/completions -H "Authorization: Bearer $SWIFTRELAY_API_KEY" -H "Content-Type: application/json" --data-binary @request.json

For a supported local-file request, replace the image URL value with a data URL structured as data:image/jpeg;base64,BASE64_ENCODED_IMAGE_BYTES. Encode the file’s actual bytes, not its filename, and match the MIME type to the image format.

5. Inspect the answer and fix image-input failures

For a successful non-streaming chat-completions response, inspect choices[0].message.content. Compare the answer with the original image before integrating it into an automated workflow.

  • Authentication error: verify the key and authorization header.
  • Unsupported image input: confirm that the exact model route accepts image parts.
  • Image retrieval error: check URL accessibility and whether the response is an image.
  • Payload too large: resize the image or reduce unnecessary detail within documented limits.

Once the first image works, improve one variable at a time: crop the relevant region, sharpen the question, or request a specific output structure. Keep the working payload as your baseline for the next lap.

Keep reading

Back for another lap?Log in