> For the complete documentation index, see [llms.txt](https://docs.artific.nl/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.artific.nl/en/for-administrators/quality/runs.md).

# Runs and results

Reading what quality control found. Found on the **Test history** tab of a quality control.

## The run list

Every run appears with its date, the assistant or element it tested, how long it took, its score and its status.

| Status                          | Meaning                           | What to do                       |
| ------------------------------- | --------------------------------- | -------------------------------- |
| **OK**                          | Everything required passed.       | Nothing.                         |
| **Warning**                     | Optional checks failed.           | Look, decide whether it matters. |
| **Issues**                      | Required checks failed.           | Investigate.                     |
| **Running**                     | In progress, with a step counter. | Wait.                            |
| **Pending**                     | Queued.                           | Wait.                            |
| **Cancelled** / **Interrupted** | Did not finish.                   | Run it again.                    |

## Reading a run

Open a run to see every scenario, expandable to its steps. Each step shows the question, the answer the assistant actually gave, what was expected, and each check with its score and the reason behind it.

The comparison between **Actual response** and **Expected** is where the useful information is. Read the actual answer properly rather than only the pass or fail. An answer can technically pass every check and still be poor.

## When a check could not be scored

Some results are not failures of the assistant but failures of the test:

| Message                                        | What it means                                            |
| ---------------------------------------------- | -------------------------------------------------------- |
| The assistant gave no answer                   | Nothing came back. Usually a model or tool problem.      |
| No response time was recorded                  | A timing check had nothing to measure.                   |
| The judge model took too long                  | The judge timed out. Run it again.                       |
| The judge model could not be reached           | A temporary problem. Run it again.                       |
| The judge model is temporarily unavailable     | A usage limit was reached. Try later.                    |
| The judge model did not return a usable result | Run it again; if it persists, simplify the judge prompt. |

A step can also be **skipped because an earlier step failed**: steps run in order, so one failure stops the rest of that scenario. Fix the first failure first; the rest are often consequences of it.

## Turning a failure into a fix

Work through it in this order:

1. **Read the actual answer.** Is it genuinely wrong, or is the check too strict? An over-strict check is the most common cause of a red run, and the fix is the check.
2. **If the answer is wrong, find out why.**
   * Missing information → the knowledge base is incomplete or the document failed to index. Check the document's status.
   * Wrong information → a superseded document is still in the collection. Remove it.
   * Right facts, wrong behaviour → the instructions. Be more specific.
   * No tool was used when one should have been → the tool description.
3. **Fix it, then re-run** to confirm.
4. **Keep the scenario.** It is now protecting you against that specific regression.

## Using it as a routine

* Switch on **Run automatically every day** for anything that matters.
* Check the results after any change to instructions, knowledge or model.
* Track the score over time. A slow decline usually means the knowledge base is drifting out of date rather than anything sudden.
* When a run goes red for a reason you decide is acceptable, change the check to match the new reality. Leaving known failures in place trains everyone to ignore the results.