> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dema.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Quality and evaluation

> How we monitor the quality of the Dema agent.

To ensure the top quality of the Dema agent, we continuosly monitor its performance in production and measure the agent's behaviour offline before any significant changes in the system.

***

## Testing before the major changes

Before going live with a large change - a new model, capability, or a change in how the agent
works - we evaluate the agent on a benchmark of scenarios that mirror how the Dema users interact with it:
asking deep analytical questions, creating apps and dashboards, maintaining longer conversations where the goal takes shape
over several messages.

We grade the responses using a set of criteria:  did the agent actually solve the problem, did it use the
correct data and produce accurate conclusions, did it follow the internal rules, and did
it stay consistent across the conversation. New models are compared side by side on identical
scenarios before we adopt them.

## Monitoring in production

Quality is also measured continuously in production, where we track the trend of the performance and investigate anything that falls short, ensuring the high quality of the platform.

<Note>
  If an answer ever looks inaccurate, use the thumbs down on the message. The feedback is stored with the
  conversation and helps the team find cases to review.
</Note>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.