A useful internal search tool does not need to become a company-wide knowledge platform. Start with one collection, one audience, and one clear job: finding the document that answers a recurring question. Current good practice favors a small, testable retrieval pipeline over an elaborate agent that can browse everything. Here is a practical checklist for getting that first version into use.
Pick one collection and define a useful result
Choose a bounded source, such as engineering runbooks or support procedures. Interview the people who search it and collect representative questions, including exact identifiers, vague descriptions, and questions the collection cannot answer. Decide whether success means returning a document, locating a passage, or producing a cited explanation. For a tiny tool, document titles and matching snippets are often enough.
Make source links the core feature. A generated answer is an optional layer, not a substitute for opening the original document. Show the title, relevant passage, source location, and last-updated date so readers can judge whether a result still applies.
Build an index you can update and delete
Keep the ingestion path boring: fetch, normalize, split, attach metadata, and index. Preserve headings, list structure, and code blocks where possible. Split documents along meaningful boundaries rather than cutting every passage at an arbitrary character count. Include enough surrounding context to keep a procedure or exception understandable.
- Assign stable document and chunk identifiers.
- Store source location, owner, modification time, and access metadata.
- Track content hashes so unchanged documents do not need reprocessing.
- Propagate deletions and permission changes, not just new content.
Separate your canonical document store from the search index. That makes reindexing, changing chunk boundaries, and investigating stale results much easier.
Start with keyword search and add semantics deliberately
Keyword retrieval remains valuable for error messages, ticket identifiers, product names, and exact configuration keys. Use your existing database search capabilities if they meet the need. Add vector retrieval when real queries reveal vocabulary mismatches, such as someone searching for a concept that the document describes with different words.
If you combine keyword and vector results, use a rank-fusion method rather than adding unrelated raw scores. Retrieve a small candidate set and consider reranking only when evaluation shows a benefit. Choose an embedding service separately and verify its data-handling terms; chat-model compatibility does not imply embedding support.
Apply permissions before content reaches the model
Authenticate the user and enforce document access during retrieval, before snippets enter a prompt. Use server-derived identity and group membership, not permissions supplied by the browser. Include access scope in cache keys, and prevent titles, snippets, counts, and logs from exposing restricted material.
For answer generation, treat retrieved text as untrusted evidence. A document can contain instructions that attempt to redirect the model. Keep tool permissions narrow, separate source material from application instructions, and never let retrieved content authorize actions. Prompt wording helps, but application-level controls remain the security boundary.
Add cited answers behind a small model adapter
Ask the model to answer only from the supplied passages, cite their identifiers, and say when evidence is missing or conflicting. Validate that returned citations reference retrieved sources. This checks citation integrity, not whether each claim is actually supported, so keep inspecting answer quality.
SwiftRelay is a practical option for this optional answer layer: it provides pay-as-you-go access through OpenAI-compatible SDKs, letting you keep provider details behind a small adapter. Claude Haiku 4.5 is one candidate to evaluate for concise, grounded responses. Sign up for SwiftRelay to use the small gift credit for a trial against your own search questions.
Check retrieval quality before widening the rollout
Maintain a compact evaluation set with expected documents, unanswerable questions, stale-content cases, and cross-permission tests. Assess retrieval separately from answer generation: a polished answer cannot rescue missing evidence. Record which results users open and offer a simple way to report irrelevant or outdated content.
Log document identifiers and operational errors without retaining sensitive queries or passages unnecessarily. Set token budgets, cap retrieved context, and monitor spending. Expand the collection only after updates, deletions, permissions, and failure cases behave predictably. That is a stronger foundation than adding more model orchestration.