Workbench

Choose your parameters for controlling AI models, and test using Score, Compare, and Panel Evaluation.

The Workbench is OWL's main interface for controlling and testing LLMs. It opens when you enter the OWL app, and you can return to it from WORKBENCH in the top menu.

Set your controls

The top control row sets how the model responds:

Control	What it does
Model	Selects the LLM that generates the response.
Portfolio	Selects the portfolio the model draws on for its answers.
Constitution	Selects the constitution that governs the model's identity, values, prohibited behaviors, and output style.
Score	Turns on scoring for your response.
Compare	Runs your output setup against a second setup.
Advanced Settings (gear icon)	Controls how the model retrieves and uses portfolio content. See below.

Enter a prompt

Select a test from My Tests or Public Tests, or type your own prompt in the text box.
To edit a selected test, click Customize as Freeform. To remove it, click Clear.
Click the send button.

After the response arrives, you can ask follow-up questions in the box below it. Click New Chat to start over.

Advanced Settings (Gear Icon)

These control how your portfolio shapes the model's answers. OWL uses retrieval-augmented generation (RAG): when you submit a prompt, OWL retrieves the most relevant passages (RAG snippets) from your portfolio and passes them to the model with your prompt. This grounds the model's output in your approved sources, so it relies less on general training data.

Max Sources sets how many RAG snippets OWL retrieves for each prompt.

A lower number limits the model to the most relevant passages. Use this for precise, tightly grounded answers.
A higher number gives the model a broader range of content. Use this when a prompt draws on several topics.

Meta-Prompt holds the instructions that tell the model how to use the RAG snippets. The default meta-prompt directs the model to:

Treat portfolio content as accurate unless its own knowledge clearly contradicts it
Reflect the values and priorities in the portfolio content
Use its own knowledge where the portfolio doesn't cover the question

Evaluating Model Behavior

OWL allows you to evaluate model behavior against values that you select, using tests that you select.

OWL provides three ways to evaluate model performance:

Score – Instant evaluation of responses against configured evaluation dimensions.
Compare – Side-by-side comparative evaluation between two setups with score comparisons and an output summary.
Panel Evaluation – Comprehensive batch evaluation testing a target model configuration across multiple evaluator judge models and tests simultaneously.

Score

Score evaluates your response against the evaluation dimensions in your application settings, which should be aligned with the values in your Constitution. A new interface is coming soon to customize this.

Turn on scoring

Set your Model, Portfolio, and Constitution.
Turn on the Score toggle.
Enter a prompt or select a test, then click the send button.

Read the results

The response card shows the Average Score in the top right corner. Below the response, Evaluation Scores list each dimension with:

A score from -2 to +2
A one-line rationale for the score

Hover over any rationale to read it in full. Click Query Statistics to view details about the run.

Note: Follow-up questions are disabled for scored runs. Click New Chat to start a new query.

Compare

Compare runs the same prompt through two setups side by side, so you can see how your portfolio, constitution, and/or model choice changes the answer.

Set up a comparison

Set your Model, Portfolio, and Constitution. This is Setup 1.
Turn on the Compare toggle.
Under Compare Target (Setup 2), choose one:
- Base Model (no RAG · no constitution): compares against the same model with no portfolio or constitution applied.
- Custom OWL Setup: opens Model 2, Portfolio 2, and Constitution 2 so you can configure a second setup.
Enter a prompt or select a test, then click the send button.
Review the confirmation, which names both setups and notes that the comparison doubles model calls, cost, and latency. Click Run anyway to start.

Each comparison costs 2 credits per query.

Note: The two setups must differ in at least one parameter. If they match, OWL blocks the run and shows an error next to the send button.

Read the results

The results show two answer cards side by side. Each card displays its setup label (for example, OWL Setup 1, OWL Setup 2, or Base model), the setup details, the full response, its sources, and its average score.

Below each response, Evaluation Scores shows the same dimensions for both sides, so you can compare them directly. Hover over any rationale to read it in full.

The Output summary lists:

Where they agree
Where they differ

OWL saves every comparison, including both answers, setups, and scores, under the name you provide. Find saved comparisons in Evaluation History.

Note: Follow-up prompts are disabled after a comparison.

Panel Evaluation

Panel Evaluation runs one or more test prompts against multiple evaluator models at once. Use it to get a broader read on quality than a single judge model provides.

Run a panel evaluation

Set your Model, Portfolio, and Constitution. These form the Target Setup.
Click Panel evaluation.
Under Evaluator LLM Models, select 1 to 4 models to act as judges.
Under Prompts, select 1 to 10 tests from your test library.
Click Start panel evaluation.
Review the confirmation, which lists the target model, portfolio, constitution, evaluator models, and total evaluation tasks (tests × evaluators). Click Confirm & Start Run to begin, or Back to change selections.

Read the results

The Panel Evaluation page shows:

Run details: who started it, date and time, portfolio, target model, and evaluator models
Overall Score: the mean across all prompts
Query Results: one row per prompt with its evaluators and score. Click View for full details.

The Panel Evaluation Results page for each prompt includes:

Section	Contents
Query Information	Date and time, LLM provider, portfolio, test, vector index, and constitution
Your Prompt	The full prompt text
LLM Response	The full response. Click the arrow to collapse or expand it.
Evaluation	One row per evaluator model, with the overall score and a score and rationale for each dimension. Hover over a rationale to read it in full.
RAG Snippets	Each source the model retrieved, with its relevance score. Click the arrow to expand a snippet.
Statistics	RAG snippet count, RAG duration, LLM duration, response length, and prompt length

If an evaluation fails, OWL flags the error on the results page.

Next: Portfolios