A documentation translation shortlist should answer a practical question: which model produces text your reviewers can approve without repairing technical meaning or document structure? Start with two candidates, a representative sample, and explicit acceptance rules. This first run gives you evidence for a choice without turning model selection into a separate project.
1. Define what documentation translation must preserve
Before choosing a model, list the parts of your documentation that must survive translation unchanged. Separate reader-facing prose from executable content and identifiers. A fluent sentence is still a failure if it changes a command, weakens a warning, or invents a product capability.
- Translate: explanatory paragraphs, descriptive headings, and reader-facing table text.
- Preserve: command names, API fields, environment variables, file paths, and code blocks unless explicitly marked for translation.
- Control: product terminology, tone, capitalization, and references to interface labels.
Decide separately whether code comments, image descriptions, and link text belong in scope. Record those decisions so reviewers do not apply conflicting standards.
2. Assemble a small, difficult documentation sample
Select excerpts from a quick-start guide, an API reference, a troubleshooting page, and a safety-sensitive warning. Include a nested list, a table, an internal link, and a code example. These expose different failure modes that a clean introductory paragraph will miss.
Use source material your team understands well. Keep an approved terminology sheet beside it, including preferred translations and terms that must remain unchanged. If you lack an approved target-language version, arrange review by someone who understands both the language and the product.
3. Put two models on the translation shortlist
Use Claude Sonnet 5.5 and Gemini 3.1 Pro as candidates for this first evaluation. Treat them as options to test, not as a predetermined ranking. Your terminology, source quality, and document structure should determine the winner.
SwiftRelay is useful here because it lets you try both through one pay-as-you-go relay, with per-token billing. Sign up for SwiftRelay to run the same documentation sample through both candidates. Keep the initial scope narrow so you can inspect every output rather than accumulate translations nobody has reviewed.
4. Give both candidates the same translation contract
Write one reusable instruction containing the source language, target language, audience, glossary, and preservation rules. Specify the output format and prohibit added explanations. Provide each document excerpt with enough surrounding context to resolve ambiguous terms.
A practical instruction is: Translate the supplied documentation into the specified target language for software developers. Follow the glossary. Preserve code blocks, identifiers, URLs, and document structure. Do not add facts, omit warnings, or soften requirements. Return only the translated document.
Attach the actual glossary and target locale to each request. Explicit regional conventions are more useful than asking for a generally natural translation.
5. Review meaning and structure before judging style
Hide model names during human review where practical. Check technical accuracy first, terminology second, structural preservation third, and readability last. This prevents polished prose from concealing a changed instruction.
- Meaning: prerequisites, permissions, warnings, and optional steps retain their original force.
- Consistency: recurring concepts follow the approved glossary.
- Integrity: links, placeholders, code fences, and list nesting remain valid.
- Readability: sentences sound natural without expanding the product’s claims.
Record defects and the edits needed to fix them. Distinguish critical errors that block publication from stylistic preferences that a reviewer can resolve.
6. Connect the selected candidate to your publishing workflow
If your pipeline uses an OpenAI-compatible SDK, configure its base URL as https://api.fastapirelay.com/v1. Keep credentials in environment variables rather than documentation files. Store prompts and glossary versions alongside translation jobs so failures can be reproduced.
Translate by logical section, carrying relevant context into each request. Validate protected content automatically, then inspect the rendered documentation. Syntactically valid output can still produce broken navigation or misleading layout.
7. Approve a default and keep the test reusable
Select the candidate with the most acceptable results and manageable review effort on your sample. Keep the other as a fallback only if it passes the same acceptance rules. Save the sample as a regression set, and rerun it whenever you change models, prompts, terminology, or documentation tooling.