Pin down the job before the first question
Interview questions do not start as questions. They start as a clear picture of the work. Before you draft anything, write down what the person will actually do in their first six months. List the systems they will touch, the decisions they will own, and the kinds of problems that land on their desk. Keep this in plain language, the kind of language you would use with a friend outside the industry.
Next, separate what the role truly requires from what would merely be nice to have. A junior backend role may need comfort with databases and code review, while knowledge of container tooling can wait. This split matters because every requirement turns into at least one question later. If you cannot explain why a skill is required, drop it from the list.
- The tasks that fill most of a typical week.
- The decisions the person makes without asking for approval.
- The people they work with and the handoffs they manage.
- The signals that tell you they are doing well after three months.
This short brief becomes your reference for the whole draft. When someone suggests adding a clever puzzle question, you can check it against the brief and keep only what predicts real performance. New interviewers often skip this step and end up with questions that sound impressive but tell them nothing about the job.
Map every question to a task on the job
With the brief in hand, turn each requirement into a question that tests the behavior behind it. Ask yourself what you would see or hear if the candidate really had the skill. A prompt about a time a database change caused trouble and what they did beats a simple yes-or-no question about database knowledge, because it asks for a story you can examine from several angles.
Group your questions into blocks: one block for technical judgment, one for day-to-day collaboration, one for past projects. Inside each block, order prompts from easier to harder so the candidate warms up before you reach the demanding ones. This also keeps the interview feeling like a conversation rather than a quiz.
Write prompts that have a shape: situation, action, result. Then decide in advance what a strong, middling, and weak answer would sound like. That foresight keeps you from judging a candidate on charisma, shared hobbies, or how much they remind you of yourself.
Aim for a small set of questions you can actually finish. Eight solid prompts that invite real detail will teach you far more than twenty shallow ones rushed through in the final ten minutes.
Score answers against a simple checklist
A question without a scoring method is a memory test for the interviewer. Write down the signals you want before the interview starts, and score each answer as you go. Keep the scale small, three or four levels at most, so you are not inventing fine distinctions under pressure.
Give each question two or three concrete signals to look for, then describe what good evidence looks like. The signals change with the question, but for technical judgment prompts you might watch for:
- What the candidate noticed first when they saw the problem.
- The reasoning they used to choose one fix over another.
- How they weighed trade-offs such as speed, risk, and clarity.
- Whether they followed up after shipping to check the result.
Share the checklist with everyone interviewing for the role before the first candidate walks in. When two interviewers use the same signals, debriefs get shorter and fairer. It also surfaces bias quickly, because a score that disagrees with the written signals has to be explained.
Send the draft through a quick pit stop
Fresh eyes catch bad questions. Before you use the set, read every prompt aloud and try answering it yourself. If your own answer wanders or the prompt needs a paragraph of setup, the candidate will struggle too.
Check for the usual traps, then trim. A short pass through the list is enough:
- Cut anything that tests memorized trivia unrelated to daily work.
- Split questions that ask for two answers at once.
- Replace jargon with the words your team actually uses.
- Add a note on how much time each block should take.
Then run the set on a colleague or a recent hire and see which prompts produce stories and which produce shrugs. The shrugs are the ones to rewrite. Repeat this pass every few hiring rounds, because the job changes and stale questions quietly stop predicting anything.
Speed up the repetitive drafting with a model
Once the structure is set, drafting variants becomes repetitive work. A model can generate paraphrases, answer rubrics, and follow-up prompts while you keep editorial control. Claude Haiku 4.5 is handy for quick bulk variations, while Claude Opus 5.5 helps when you want careful, nuanced scoring notes.
SwiftRelay makes this cheap to try. It is a pay-as-you-go relay for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and other models, with billing per token and no subscription, so you pay only for the tokens a drafting session consumes. It works with Claude Code, Codex, Gemini CLI, and any OpenAI-compatible SDK. Claude Code and Gemini CLI take the bare host, https://api.fastapirelay.com, and OpenAI SDKs add /v1 to that base.
Treat model output as a draft rather than a verdict. Keep the prompts that match your brief, rewrite anything that drifts into generic interview clichés, and always read the scoring notes before trusting them. A model is fast at producing options, but only you know what good engineering looks like on your team.
Sign up for SwiftRelay and use the small gift credit on new accounts to run your first question set, and if the workflow sticks, inviting friends earns 14% of their top-ups.