> For the complete documentation index, see [llms.txt](https://docs.artific.nl/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.artific.nl/en/for-administrators/quality.md).

# Quality control

Automated checks that an assistant or a Toolbox element keeps giving good answers. Found under **AI assistants → Quality control** and **AI Toolbox → Quality control**.

## Why it exists

An assistant is not a fixed thing. You change its instructions, someone updates a document, a model is upgraded, and an answer that was right last month is now wrong. Nobody notices, because nobody re-asks the questions that used to work.

Quality control is the regression test for that. You write down the questions that matter and what a good answer looks like, and the platform re-checks them on demand or every day.

## How it is built

| Concept             | What it is                                                           |
| ------------------- | -------------------------------------------------------------------- |
| **Quality control** | A named set of checks, linked to one or more assistants or elements. |
| **Scenario**        | One situation to test, made of one or more steps.                    |
| **Step**            | A question, and the checks its answer must pass.                     |
| **Check**           | A single condition, such as "must contain this text".                |
| **Run**             | One execution of everything, producing a score.                      |

## Setting one up

1. Go to **Quality control** and choose **Add quality control**.
2. Give it a **Title** and **Description**, choose a **Judge model**, and link the assistants or elements it applies to.
3. Optionally switch on **Run automatically every day**.
4. Add [scenarios and steps](/en/for-administrators/quality/scenarios.md).
5. Run it, and read the [results](/en/for-administrators/quality/runs.md).

You can also add quality control from an individual assistant's **Quality control** tab, which pre-links that assistant. There, **Add quality control** offers **Create new** or **Reuse existing**. The latter links a set of checks you have already written to this assistant as well, which is how you apply one standard across several assistants.

## The judge model

Some checks are mechanical: does the answer contain this phrase, did it respond in time. Others need judgement: is this answer actually correct, is the tone right. Those are assessed by a second model: the **Judge model**.

Pick a capable model here. The judge's job is harder than the assistant's: it has to evaluate rather than generate.

## What to test

Start with the questions where a wrong answer costs something:

* **The most frequent questions.** Read [session logs](/en/for-administrators/assistants/session-logs.md) to find out what they actually are.
* **Questions with one correct answer**: a price, a deadline, a policy limit.
* **Questions the assistant should refuse**, to confirm it still refuses them.
* **Questions that need a tool**, to confirm the tool still fires.
* **Anything that has gone wrong before.** Every bug you fix should become a scenario, so it cannot come back unnoticed.

Ten well-chosen scenarios are worth more than a hundred vague ones.

## Running it

**Run all** runs every quality control. An individual one has its own **Run quality control** button. Scenarios run in parallel; the steps within a scenario run in order.

Switch on the daily schedule for anything that matters. Checks you have to remember to run are checks that do not get run.

## Reading the result

Each run gets a score and a status:

| Status      | Meaning                                     |
| ----------- | ------------------------------------------- |
| **OK**      | Everything passed.                          |
| **Warning** | Optional checks failed. Worth a look.       |
| **Issues**  | Required checks failed. Something is wrong. |

See [Runs and results](/en/for-administrators/quality/runs.md) for how to read the detail.