> ## Documentation Index
> Fetch the complete documentation index at: https://critiqor-71f5274a.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Best Practices for Getting Reliable Critiqor Evaluations
> Get more accurate Critiqor evaluations by choosing realistic workloads, running multiple sessions, testing edge cases, and acting on recommendations.
Critiqor evaluations are only as useful as the sessions that generate them. A session with trivial prompts and no edge cases will produce a limited signal. A session that reflects your agent's real workload — with varied inputs, realistic tasks, and deliberate stress testing — produces an evaluation you can act on. Follow these practices to consistently get accurate, meaningful results.
Test with prompts and tasks that reflect how the agent is actually used — not toy examples.
One run is a single data point. Multiple runs across varied scenarios give you a reliable picture.
After every meaningful change, run a new session and compare it to your baseline.
The Recommendations section is your improvement checklist. Work through it in severity order.
Deliberately stress your agent with ambiguous inputs and failure scenarios for maximum signal.
Treat Critiqor as a required review step before releasing agent changes to production.
***
## Observe realistic workloads
The quality of your evaluation depends directly on the quality of your session. Use prompts, tasks, and interaction patterns that mirror real production usage. If your agent handles customer support queries, run customer support queries. If it performs multi-step research tasks, run those.
Minimal or trivial prompts — simple greetings, single-word queries, or copy-pasted boilerplate — produce limited signal and can make a fragile agent appear more reliable than it is.
**What good session coverage looks like:**
* A variety of prompt types, not just your most common case
* Prompts of different lengths and complexity levels
* Tasks that exercise different tools or capabilities your agent relies on
* At least a few prompts that are intentionally harder or more ambiguous
***
## Run multiple sessions
A single observation run gives you one data point. Trust scores and diagnoses can vary across sessions depending on the specific prompts and conditions you tested. Running multiple sessions — especially across different prompt sets and use cases — gives Critiqor enough evidence to produce a stable, confident reliability picture.
Start by establishing a baseline with your current agent. Then re-run sessions whenever you make a meaningful change. Over time, you'll build a history of runs that makes trends immediately visible.
***
## Compare runs over time
After every meaningful change to your agent — a prompt update, a new tool integration, a model version bump, a config tweak — run a new observation session and compare the results to your previous runs.
Use `critiqor runs` to list your full evaluation history, then open specific runs side-by-side:
```bash theme={null}
critiqor dashboard run_005
critiqor dashboard run_003
```
Look for changes in trust score, shifts in the Primary Diagnosis, and whether previously flagged issues from the Recommendations section have been resolved. Consistent improvement across runs is a strong signal your changes are working.
***
## Act on recommendations
Every evaluation includes a Recommendations section in the dashboard — a prioritized list of specific, actionable improvements based on what Critiqor observed. Treat this as your improvement checklist, not background reading.
Work through recommendations in order of severity. Higher-severity findings have more impact on your trust score and reliability in production. After addressing a finding, run a new session to confirm the improvement was effective before moving on.
Ignoring recommendations and re-running sessions will show little or no improvement in your trust score — Critiqor will continue to surface the same issues until they're resolved.
***
## Test edge cases
Your agent's weakest points are rarely exposed by your most common, well-formed prompts. Deliberately test the scenarios where it's most likely to struggle:
* **Ambiguous inputs** — questions that are underspecified or have multiple valid interpretations
* **Tool failures** — what happens when an external tool returns an error or unexpected output?
* **Long sessions** — extended conversations or multi-step tasks that accumulate context
* **Out-of-scope requests** — prompts that push the agent beyond its intended use case
* **Conflicting instructions** — inputs that create tension between goals or constraints
Edge-case sessions tend to produce the most diagnostic signal. They surface issues that wouldn't appear under normal conditions — exactly the issues that matter most before a production release.
***
## Review before promoting
Treat Critiqor as a required step in your release process, not an optional audit after the fact. Before promoting any agent change — to staging, to a wider audience, or to production — run an observation session and review the evaluation.
If your trust level is **Low** or **Moderate**, investigate the Primary Diagnosis and Recommendations before releasing. A Low trust level in particular warrants a close review: it indicates reliability issues significant enough to affect real users.
Use the `critiqor check` command to automate this gate in your deployment pipeline. See [Policy Checks](/guides/policy-checks) for details.
***
Label your sessions mentally before you run them — note what you changed and why. When you're comparing `run_005` to `run_003` two weeks later, you'll know exactly what was different between them and whether the trust score change is explained by your update.