> For the complete documentation index, see [llms.txt](https://docs.artific.nl/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.artific.nl/en/for-administrators/knowledge-base/advanced.md).

# Advanced

The machinery underneath search tools: indexes, retrieval tools and templates. Found under **Knowledge base → Advanced**.

{% hint style="info" %}
You do not need this screen for normal work. The [search tool wizard](/en/for-administrators/knowledge-base/search-tools.md) creates and configures everything here for you. Come here when you have a specific problem the guided setup cannot solve, and expect it to be technical.
{% endhint %}

## How the pieces fit

| Piece              | What it is                                                                                 |
| ------------------ | ------------------------------------------------------------------------------------------ |
| **Collection**     | Where the files live.                                                                      |
| **Index**          | The prepared, searchable copy of the documents routed into it.                             |
| **Retrieval tool** | How that index is queried. This is what an assistant actually calls.                       |
| **Template**       | A ready-made blueprint of processing steps you can load into an index or a retrieval tool. |

A search tool created through the wizard is one index plus one retrieval tool bound to it.

## Indexes

An index holds the processed copies of your documents. It is defined by which collections feed it, which rules decide what gets in, and which steps are used to prepare each document.

### Overview

| Setting                    | What it does                       |
| -------------------------- | ---------------------------------- |
| **Name**                   | Identifies the index.              |
| **Description**            | Optional.                          |
| **Collections to include** | Which collections feed this index. |

The summary cards show how many files and segments the index holds, its model, its vector size, and when it was last refreshed. The refresh control offers an **Incremental reindex** (only what changed) or a **Full reindex** (everything).

{% hint style="warning" %}
The model an index uses is fixed when the index is created and cannot be changed afterwards. To change it, create a new index and move the collections over.
{% endhint %}

### Routing rules

Rules decide which documents from the selected collections actually enter this index. Leave them empty and everything enters.

Each rule group matches **all** or **any** of its conditions, and groups are combined with OR. A condition is a field, an operator and a value:

| Field              | Matches on                                                              |
| ------------------ | ----------------------------------------------------------------------- |
| **Path**           | Where the file sits in the collection.                                  |
| **File name**      | The name of the file.                                                   |
| **File extension** | For example `pdf`.                                                      |
| **File type**      | The document format.                                                    |
| **Label**          | A [label](/en/for-administrators/knowledge-base/labels.md) on the file. |

Operators are **equals**, **is one of**, **contains** and **starts with**.

This is how you build an index over "everything in Policies labelled `public`" without splitting the collection.

### Ingest steps

The pipeline that prepares each document: reading the file, cleaning it up, splitting it into segments, and preparing those segments for search. The editor shows it as a diagram, with versions you can create, name and activate.

{% hint style="warning" %}
The pipeline warns you when it has no table-extraction step. Without it, tables inside documents are not extracted, and assistants cannot answer questions about tabular data in those files.
{% endhint %}

The settings most worth understanding:

| Step                 | Why it matters                                                                                                                                        |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Text extraction**  | The standard reader. An advanced variant does better on complex layouts and scanned-looking PDFs, at the cost of speed.                               |
| **Splitters**        | How a document is cut into segments. Segment size and overlap are the main levers: smaller segments are more precise, larger ones carry more context. |
| **Table extraction** | Pulls tables out so they can be queried as data.                                                                                                      |
| **Enrichment**       | Adds summaries, keywords or other metadata to improve retrieval.                                                                                      |

Changing ingest steps only affects documents processed afterwards. Reindex to apply them to what is already there.

### Indexed documents and history

**Files in this index** lists every document with its status, and offers **Retry failed files** when some did not make it. **Recent activity** shows past indexing runs, what triggered them, and how many documents were indexed, failed or skipped.

These two are your first stop when an assistant cannot find a document you know you uploaded.

## Retrieval tools

A retrieval tool defines *how* an index is searched. Create one with a **Name**, a **Description**, the **Index** it queries, and a **Search Type**:

| Search type  | What it does                                         |
| ------------ | ---------------------------------------------------- |
| **Semantic** | Matches on meaning.                                  |
| **Keyword**  | Matches on exact words.                              |
| **Hybrid**   | Both, combined. The usual choice.                    |
| **Image**    | Searches images. Only for an index built for images. |

The **Description** is not decoration: the assistant reads it to decide whether to use this tool. Describe what it searches and when it should be used.

Index and search type are fixed at creation. To change retrieval behaviour afterwards, edit the query pipeline.

### Try it

**Try it** runs a query against the tool and shows the matching segments with their relevance scores and sources. It is by far the fastest way to diagnose "the assistant cannot find X". If the segment does not come back here, the problem is retrieval, not the assistant.

### Query pipeline

The steps that run when a query arrives: the searches themselves, then optional reranking, filtering and query rewriting. The settings you are most likely to change:

| Setting               | Effect                                                                                                        |
| --------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Number of results** | How many segments come back from a search step. More context, more noise.                                     |
| **Reranking**         | Reorders results by relevance before they reach the assistant. Improves quality, adds latency.                |
| **Relevance filter**  | Drops weak matches instead of passing them on.                                                                |
| **Query rewriting**   | Rephrases the user's question before searching, which helps when people ask in language unlike the documents. |

## Templates

Read-only blueprints of pipeline steps, provided with the platform. Browse them under the **Templates** tab, filtered by **Ingest** or **Search**, and preview their steps.

You load a template from within a pipeline editor rather than from this screen. They are also what the platform uses when it creates a pipeline for you.