A knowledge base with cited answers
A system that collects rules, documents and news from institutional websites, indexes them and answers questions by saying which documents each statement comes from. I built it at the software company where I work as a developer, for a port authority. The authority is not named.
What it is for and what it can do
For a public body that needs to find rules and news scattered across many institutional sites. ETT offers it among its artificial intelligence solutions. It gives a single place to search, with answers that can be checked against the source.
- Collects documents and news from institutional sites and keeps them up to date.
- Combines keyword search and search by meaning.
- Answers in natural language with cited sources, to open and check.
- Caps the monthly AI spending.
- Measures answer quality on a set of test questions.
Context
People working at a port authority have to follow rules, studies and environmental news published on many different sites, in several languages and formats. The information exists but is scattered. They needed one place to search, with answers whose origin can be checked.
The project has been under development since February 2026. I wrote most of it, with one colleague.
Collecting the documents
The collector is a separate service. For each source (a website, an RSS feed, a news section) it finds pages through sitemaps and feeds and opens them with a real browser, because some sites protect themselves with JavaScript checks. It extracts the main text from the pages, while PDFs and office files go through Apache Tika.
A language model helps decide which pages to follow and which to drop as irrelevant. The instructions for each site are saved and reused, so the site is not analysed again. Collected documents arrive as drafts, and a person reviews them before publication. Collections repeat at intervals and a check avoids reprocessing pages already seen.
Searching and answering
Each published document is split into pieces of about a thousand characters, with a 150-character overlap so sentences are not cut in half. For each piece I compute a 768-value vector (produced by an embedding model, with a cache so the same text is not paid for twice) and a keyword representation (BM25). Everything is stored in Qdrant.
Search is hybrid: it merges the two rankings, so “docking” also finds “mooring” and an exact code or acronym is not lost. An optional reranking step puts the best results on top. A smaller model rewrites follow-up questions (“and that one?”) before searching, so the chat works over several turns.
The answer is written by a language model (Gemini) that receives only the retrieved pieces and must cite them by number. The system shows only the documents actually cited, and the answer is streamed.
How I evaluate it
An answer that looks right is not enough, so there is a small evaluation setup. A set of 19 questions with known answers, split into categories (keywords, meaning, several documents, filters, questions with no answer), measures whether the right documents are found (recall, precision, MRR, NDCG). Three automatic checks with a judge model then rate whether the answer is faithful to the pieces, relevant and complete. It runs the same chain as production and lets me compare two configurations, for example with and without reranking, on numbers rather than gut feeling.
I tested load with 10,137 documents (17,220 pieces in Qdrant, with random vectors to avoid spending API calls): text search goes from 160 ms to 308 ms for one user at a time. With 50 users at once the median search time is 1.9 s. The bottleneck was database connections, not the amount of data.
What I did
The search and answer service, the collector, the Django part that connects them and most of the interface, including the 3D map built with three.js. I also wrote the tests (503 in the backend suite in September 2026), the load and browser-compatibility tests and the documentation for the client. Since 2026 the service has run on a staging server of the authority.
Limits and status
The system is being tested on a staging environment and is not yet in full production. The evaluation has only 19 questions: it catches regressions, it does not give a correctness percentage. Answers depend on a paid external service, with a monthly spending cap and the option to switch provider. Extraction of video subtitles is switched off for legal caution. The project belongs to the company and is not public.
Stack
Python, Django, FastAPI, Celery, PostgreSQL with PostGIS, Qdrant, Redis, Playwright, Apache Tika, Nuxt (Vue, TypeScript), three.js, Docker.