Split the Batch Into Named Shards First
Deciding where the boundaries sit is the first gate on this checklist, because once the queue is live you should not be re-cutting work by hand. Pull the raw input apart into shards that are small enough to retry on their own and big enough to justify one request. Give every shard a stable identifier that survives a restart, and write down how many source records it covers.
Keep the shard manifest in one place: a file, a table, or a queue entry that lists the shard name, the input it covers, the model assignment, the prompt template, and the expected output shape. That manifest is the control panel for the entire run, and it is the only thing you want to read when a shard fails at an inconvenient moment.
Validate inputs before any model call. Drop records that are empty, duplicated, or malformed into a rejection file instead of letting them travel through the pipeline and poison the output. Aim for idempotent shards, where running the same shard twice still produces exactly one accepted result: write the output to a location keyed by the shard name, and set the completion marker only after the output passes a shape check.
- Stable shard IDs that point back to the exact source records.
- A rejection file for inputs that fail validation before routing.
- A dry run on a handful of shards through the real prompt and the real parser.
The dry run is where you find out that the parser expects a field the prompt never asks for. Fix that gap before the queue fills up, not after.
Assign Shard Types to DeepSeek V4 Flash and Qwen3.7 Plus
Routing is a decision, not a coin flip. Group the shards by task type first, and only then decide which model handles which group. A batch usually contains two or three recognisable task types, and pretending it is one homogeneous workload is how runs end up with inconsistent output.
DeepSeek V4 Flash fits the uniform, high-volume end of the job: extraction, classification, short structured summaries, and anything where the answer is a fixed schema and every shard runs the same instruction. Qwen3.7 Plus fits shards that need longer generation and editorial judgement: multi-paragraph drafts, rewrites with a tone brief, and answers that have to pull several source notes into one coherent passage.
Write the routing decision as a small table you can paste into the manifest. Three columns are enough.
- Shard type: the task, such as extract, classify, or draft.
- Model: DeepSeek V4 Flash or Qwen3.7 Plus, stated exactly as the model string you will send.
- Prompt template: the versioned prompt plus the output contract the parser enforces.
Keep prompts as close to identical as possible across the two models. Where behaviour differs, you want the difference to come from the model choice, not from two drifting instruction sets that nobody can compare. Version every template, and record the version in each result so a later review can trace an odd output back to the prompt that produced it.
Leave an escalation path in the plan. If one shard type keeps failing the output validator, send a sample to DeepSeek V4 Pro or Qwen3.8 Max and decide whether the whole class should move. Escalation should be a routing change in the manifest, not an improvised intervention mid-run.
Point the Client at the SwiftRelay Host
One relay key lets the same job talk to both models, which keeps the plumbing identical across shards. SwiftRelay is a pay-as-you-go relay for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and other models, and it works with Claude Code, Codex, Gemini CLI and any OpenAI-compatible SDK.
The API host is https://api.fastapirelay.com. Claude Code and Gemini CLI take the bare host, while OpenAI-compatible SDKs add /v1 to that base. Getting this wrong is the most common setup snag, so check the base URL in the client config before you write any batch code.
- Store the key outside the repo, in an environment variable the worker reads at start.
- Send one test request per model and confirm the response parses against your output contract.
- Print the model string in every log line, so a routing mistake is visible in the first minute.
Billing is per token and pay as you go, which suits a batch that is sized by the work rather than by a plan. Sign up for SwiftRelay, use the small gift credit that new accounts receive for your dry run, and keep the real budget separate. If teammates join the effort later, inviting friends earns 14 percent of their top-ups.
Run the two test calls as a pair rather than one at a time. You are checking that both models return the same output shape, so the parser written for one shard type can safely handle the other.
Add Retries, Checkpoints, and a Token Budget
Large batch jobs do not fail once; they fail in classes. Before the green light, decide what each class of failure means and what the worker does about it. A retry that repeats a bad request forever is worse than a clear stop, so cap the attempts and move the record to a review queue after the cap.
- Checkpoint after every shard, writing the completion marker only when the output has been validated.
- Retry transient failures with backoff, and treat a repeated parse failure as a prompt problem instead of a network problem.
- Set a token budget per shard type, so a runaway generation cannot consume the budget meant for the rest of the run.
- Log shard name, model, prompt version, token counts, and status for every request.
The token budget does double duty: it bounds spend and it flags the shard types whose outputs are longer than the plan assumed. When one class of shards consistently lands near its ceiling, that is a signal to tighten the prompt or move the class to the model better suited to it.
Make the run resumable by design. A worker that restarts should read the manifest, skip everything marked complete, and continue from the first unfinished shard. Test that behaviour deliberately: kill the worker mid-run during the dry pass and confirm the restart picks up cleanly without duplicating accepted results.
Run the First Lap and Check the Outputs
The first lap is a validation exercise, not the main event. Take a small set of shards covering every task type, run them end to end, and inspect the output by hand before letting the full queue loose. Compare the two models on the same shard types where the routing allowed it, and write down what you actually observed.
- Sample the output for each shard type and check it against the contract, not just against a feeling.
- Confirm the merge step assembles shard outputs in a deterministic order with no missing entries.
- Verify the audit trail, so any result can be traced to its shard, model, and prompt version.
- Re-run one completed shard to prove the idempotent path really produces one accepted result.
Green light the full run only when every checklist item above has an answer you could hand to a teammate. The manifest is complete, both test calls parsed, retries are capped, checkpoints resume, budgets are set, and the first lap showed consistent output from DeepSeek V4 Flash and Qwen3.7 Plus on their assigned shard types. Once those gates are clear, the rest of the job is mostly watching the queue drain and reviewing the sample.