pAIring knowledge base (RAG)

The knowledge base lets the AI answer from your own documents. pAIring searches them on every message, adds the best-matching passages to the prompt, and shows Sources under the reply. This is called RAG (retrieval-augmented generation). This page explains how it works, what each setting changes, and how to make it faster or more accurate.

How a question finds its answer

The work happens in two parts: once when a document is added, and again each time you send a message.

Once, when a document is added

pAIring reads the document’s text and cuts it into passages of about 500 characters. Each passage overlaps the next by 75 characters, so a sentence cut at a boundary still appears whole in one of them.

The embedding model then turns each passage into a list of numbers that represents its meaning, called a vector. Passages about similar things get similar vectors, even when they use different words. The vectors are stored with the passages, so this slow step is never repeated for the same document.

Each time you send a message

1. Your message becomes a vector. The same embedding model turns your message into numbers, the same way it did for the passages. This takes about a second.
2. The closest passages are found. pAIring compares your message’s vector with every passage you are allowed to see and ranks them by how close their meaning is. This is quick, even with thousands of documents.
3. The reranker picks the best ones (optional). Closeness of meaning is a good first filter, but it can be fooled by a passage that only sounds related. If a reranker is turned on, the closest passages (Passages to rerank, 20 by default) go to the reranker model. It reads your question together with each passage, the way a person would, and orders them by how well they answer it. This gives better results but is the slowest step on a CPU.
4. The best passages go into the prompt. The top few (Document search results, 4 by default) are placed just before your message, and the chat model reads them before it replies. The reply lists their files under Sources, so you can check where an answer came from.

Every setting on this page changes one of these steps. The settings table shows which.

Who sees which documents

Where you chat	Documents searched
A personal chat	Your own documents (Settings → Knowledge) plus shared ones
A channel	Shared documents only, so one member’s private files never reach others

Admins add shared documents. An admin can also switch the knowledge base off for everyone, or limit file size and storage. See Admin settings.

Supported files: .txt, .md, .pdf, .docx, .odt, .rtf, .html, .csv and .json. A scanned PDF with no text layer has nothing to search.

The settings

Setting	Where	Default	Faster when	Better answers when
Document search results	Each chat’s generation settings	4 (1 to 10)	Lower: fewer passages to read	Higher: more context, if the answer is spread out
Embedding model	Admin settings → System → Default models	The first installed	A small model embeds faster	A larger or multilingual model matches meaning better
Reranker	Admin settings → System → Reranker	On once a reranker is installed	Off	On, especially with many similar documents
Passages to rerank	Same panel, shown when a reranker is chosen	20 (5 to 50)	Lower	Higher, in a large knowledge base
Max loaded models	Admin settings → Inference engines → Matricxon	2	Counts chat models only, so searching never unloads the chat model	(no effect)

Install a reranker under Settings → Models → Reranker models. Reranking works with the Matricxon and llama.cpp engines; the cloud connectors don’t offer it.

Where the time goes

Each knowledge base answer has four costs. The first two are small; the last two depend on your hardware and settings.

Step	What it costs	Grows with
Embed your message	Usually under a second	Message length
Search	Milliseconds for a normal collection; about 0.1 s at 40,000 passages	Number of passages
Rerank	From seconds on a GPU to minutes on an old CPU	Passages to rerank × passage length
Read the prompt	Every added passage is read by the chat model before it replies	Document search results × passage length

As an example, on a 2011 laptop CPU without a GPU, reranking 15 passages of Hebrew text took about two minutes, and the chat model read about 3 to 6 tokens a second. On a modern CPU or a GPU, both are many times faster.

Some languages cost more

A 500-character passage is roughly 125 tokens in English, but Hebrew, Arabic and other non-Latin scripts can take 3 to 4 times as many. Each extra search result and each reranked passage costs more in those languages, so lower both on slow hardware.

Speed or better answers: what to choose

Your situation	Suggested settings
CPU only, a few documents	Reranker off, or Passages to rerank 5 to 8. Document search results 3 to 4.
CPU only, many similar documents	Reranker on with 8 to 12 passages, and expect a wait before each answer.
A GPU, a large knowledge base	Reranker on, Passages to rerank 20 to 30, Document search results 4 to 6.
Answers miss information you know is there	Raise Document search results, turn the reranker on, and ask with the document’s own words.
You need the whole document (a summary, a comparison)	Attach the file to your message instead (below).

On Matricxon, Max loaded models counts chat models only. The embedding and reranker models load beside the chat model without unloading it, so the chat keeps its saved prompt and does not re-read the whole conversation after a search. Raise it only when people chat with several different models at once.

Knowledge base or attachment?

	Knowledge base	Attachment
What the model reads	The few best-matching passages	The whole file
Speed per message	Fast: a few hundred tokens	Slow for a long file: every word is read
Later messages	Searched again on every message	Only the message it was attached to
Best for	Looking things up, many questions over time	Summaries, comparisons, anything needing all of the text

See Chat and channels for attachment limits.

Changing models

• Turning the reranker on or off, or changing it, needs no re-embedding: it reads the stored passage text. The next question uses it.
• Changing the embedding model does. Vectors from different models can’t be compared, so documents embedded with the old model are left out of searches until they are embedded again. Press Re-embed all documents under Admin settings → System → RAG. It covers every user’s documents and the shared ones, and without a GPU it can take a long time.

Very large knowledge bases

Search keeps every passage’s vector in memory, about 3 KB per passage: around 120 MB for a thousand documents of 40 passages each. That is fast and fine for most installations. A collection of hundreds of thousands of passages uses gigabytes of RAM, so keep the knowledge base focused on documents people actually ask about.

Troubleshooting

What you see	Likely cause and fix
No Sources under answers	The knowledge base is off, no embedding model is installed, or (in a channel) the document is private.
A document is never found	It was embedded with an earlier embedding model: re-embed it. Or it is a scanned PDF without text.
The wrong passage is used	Turn the reranker on, raise Passages to rerank or Document search results, or ask with words from the document.
A long wait before the answer starts	Reranking or reading the passages on a slow CPU. Lower Passages to rerank and Document search results, or turn the reranker off.
Every message re-reads the whole chat	The chat model was unloaded, usually because people use more chat models than Max loaded models allows. Raise it if there is RAM to spare.

To see where the time goes, open Stats → Visual chat workflow while a chat runs (with the Matricxon engine). It shows each step live, including embedding, reranking and reading the prompt. See Stats and monitoring.