SwiftRelay

← All sprint notes

First Lap: Build a Tiny Internal Search Tool

First lapsEN

A tiny internal search tool should solve one concrete problem: finding the document that answers a recurring team question. Start with a small collection of runbooks, onboarding notes, and project decisions. Build reliable retrieval first, then add generated answers only when the underlying results deserve trust.

1. Pick one document collection and ten real questions

Choose a folder your team already maintains rather than attempting to index every workspace. Ask colleagues for ten questions they recently answered by hunting through documents. Include exact names, unfamiliar terminology, and a question whose answer does not exist.

For each question, record the expected document and a relevant passage. This becomes your acceptance checklist. Define the first release as a search box, ranked results, readable excerpts, and a way to open the original document.

2. Create a document table and a full-text index

Use Python and SQLite with FTS5 enabled. Confirm FTS5 support in your deployment environment before building around it. Keep the initial service and database on one machine; a separate search cluster adds little value at this stage.

Create a documents table with id, title, source_url, body, content_hash, updated_at, and access_group. Store normalized text rather than raw HTML. Then create a full-text table with CREATE VIRTUAL TABLE document_search USING fts5(title, body). Use each document ID as its corresponding full-text rowid.

Keep the ordinary table authoritative. Update its row and the matching search row within the same transaction so metadata and indexed content cannot drift apart.

3. Build an importer you can run repeatedly

Read Markdown and plain-text files first. Extract a title from the first heading, preserve paragraph breaks, and assign each source a stable identifier. Compute a hash of the normalized content so unchanged files do not need replacement.

For changed documents, replace both stored text and indexed text. Track which sources appeared during a completed import, then remove records for deleted sources. Do not perform deletion cleanup after a partial or failed scan.

Exclude secrets, credentials, and private personnel records. Search indexing makes forgotten content discoverable, so review the source collection before importing it.

4. Add authenticated search with permission filters

Create a GET /search endpoint accepting a query string. Authenticate the caller and obtain their allowed access groups from your identity system, never from a browser-supplied group parameter.

Join document_search to documents, apply the full-text match and access-group restriction in the same query, order by bm25(document_search), and return up to ten results. Lower BM25 scores rank first in SQLite FTS5. Use bound parameters for values.

Bound parameters prevent SQL injection, but they do not disable FTS query syntax. For the first release, accept plain search terms, encode each as a quoted FTS phrase with embedded quotes escaped, and join them with AND. Reject empty or oversized queries and handle malformed input without exposing database errors.

5. Make each result useful without an AI answer

Return the title, source location, updated date, and a matching excerpt. SQLite's snippet function can supply excerpts; request plain-text markers and let the interface render emphasis safely. Escape document content rather than inserting source text as HTML.

Show an explicit no-results message with suggestions to shorten the query. Avoid silently replacing an unsuccessful search with a confident generated answer. Recheck authorization whenever your service serves document content.

6. Add optional answers through SwiftRelay

Once retrieval works, add an answer button that sends only authorized excerpts to a model. SwiftRelay fits this small experiment: it provides per-token billing and works with an OpenAI-compatible SDK. Set the SDK base URL to https://api.fastapirelay.com/v1, keep the API key server-side, and configure an available model identifier for Claude Haiku 4.5.

Label excerpts with source IDs. Instruct the model to treat document text as untrusted evidence, ignore embedded instructions, cite supporting IDs, and say when evidence is insufficient. Validate returned citations against the supplied sources. Sign up for SwiftRelay to try this optional layer with the small gift credit available to new accounts.

7. Run the acceptance checklist before sharing

Repeat your ten questions and inspect retrieval separately from generated answers. Test revoked access, deleted files, empty queries, and documents containing hostile instructions. Keep logs free of document bodies and credentials. Ship when teammates can find the right source and unauthorized content stays out of results, excerpts, and model requests.

Keep reading

Back for another lap?Log in