> ## Documentation Index
> Fetch the complete documentation index at: https://scorecard-d65b5e8a-update-playground-docs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Analysis

> Compare and analyze performance across multiple runs to identify improvements and regressions in your AI systems.

export const DarkLightImage = ({lightSrc, caption, alt, darkSrc = null, width = "1000"}) => {
  const getAbsoluteUrl = src => {
    if (src.startsWith('http://') || src.startsWith('https://')) {
      return src;
    }
    const currentUrl = typeof window !== 'undefined' ? window.location.origin : '';
    if (currentUrl.includes('.mintlify.app')) {
      const subdomain = currentUrl.split('.')[0].replace('https://', '');
      return `https://mintlify.s3.us-west-1.amazonaws.com/${subdomain}${src.startsWith('/') ? '' : '/'}${src}`;
    } else if (currentUrl === 'https://docs.scorecard.io') {
      return `https://mintlify.s3.us-west-1.amazonaws.com/scorecard-d65b5e8a${src.startsWith('/') ? '' : '/'}${src}`;
    } else {
      return `${currentUrl}${src.startsWith('/') ? '' : '/'}${src}`;
    }
  };
  const content = <>
      <img className="block dark:hidden" width={width} src={getAbsoluteUrl(lightSrc)} alt={alt} />
      <img className="hidden dark:block" width={width} src={getAbsoluteUrl(darkSrc || lightSrc.replace('light', 'dark'))} alt={alt} />
    </>;
  if (caption) {
    return <Frame caption={caption}>{content}</Frame>;
  } else {
    return content;
  }
};

The **Analysis** page provides a powerful interface for comparing performance metrics across multiple runs side-by-side. It enables you to visualize score distributions, examine individual test records in detail, and identify patterns in your AI system's behavior.

<DarkLightImage lightSrc="/images/analysis-overview-light.png" darkSrc="/images/analysis-overview-dark.png" caption="Analysis page showing multi-run comparison with score distributions" alt="Screenshot of the Analysis page displaying multiple runs and their score distributions" />

## What is Analysis?

Analysis in Scorecard allows you to:

* **Compare Multiple Runs**: View up to multiple runs simultaneously to track performance evolution
* **Visualize Score Distributions**: See how scores are distributed across different metrics and runs
* **Inspect Individual Records**: Drill down into specific test cases to understand outputs and scoring details
* **Identify Patterns**: Spot improvements, regressions, and consistency issues across iterations

## Navigating to Analysis

You can access the Analysis page from the Runs list page by selecting the runs you want to compare and clicking the **Analyze** button.

## Understanding Score Distributions

The score distribution matrix provides an at-a-glance view of performance across your selected runs and metrics.

<DarkLightImage lightSrc="/images/analysis-score-matrix-light.png" darkSrc="/images/analysis-score-matrix-dark.png" caption="Score distribution matrix showing performance across runs and metrics" alt="Screenshot of the score distribution matrix with color-coded performance bars" />

The matrix displays runs along the rows and metrics along the columns.

The cell shows the score distribution of that run for that metric. Different metric types are displayed appropriately. Boolean metrics show their pass rate and integer and float metrics show their average score.

## Detailed Records Table

Below the score distribution matrix, the **Records Table** provides granular inspection of individual test cases. The records table here is organized the same way as the run details page.

<DarkLightImage lightSrc="/images/analysis-records-table-light.png" darkSrc="/images/analysis-records-table-dark.png" caption="Detailed records table with side-by-side comparison" alt="Screenshot of the records table showing test case details across multiple runs" />

### Record Deduplication

The table intelligently groups records by testcase:

* **With Testcase IDs**: Multiple runs evaluating the same testcase appear in a single row
* **Without Testcase IDs**: Each record appears as a separate row
* **Visual Indicators**: Color-coded bars show which runs evaluated each testcase

## Best Practices

<AccordionGroup>
  <Accordion title="Start with Recent Runs">
    Begin your analysis with the most recent runs to understand current performance, then add historical runs for trend analysis.
  </Accordion>

  <Accordion title="Compare Runs that use the same testcases">
    If your runs have the same testcases, Scorecard automatically groups records by testcase for you.
  </Accordion>

  <Accordion title="Filter records">
    In the records table, you can filter records to only show those that mention a keyword in the inputs, outputs, or expected outputs.
  </Accordion>

  <Accordion title="Examine Outliers">
    Pay special attention to testcases where scores vary significantly between runs. These often reveal important insights. Is it a challenging or incorrect testcase? Are your metric guidelines underspecified? Has your system regressed in a particular area?
  </Accordion>

  <Accordion title="Document Findings with Run Notes">
    Use [run notes](/features/runs#run-notes) to document what you learn from analysis sessions for future reference.
  </Accordion>
</AccordionGroup>

## Common Use Cases

### Performance Regression Detection

Quickly identify when changes to your system have negatively impacted performance:

1. Select your current production run as baseline.
2. Add the latest development run for comparison.
3. Look for metrics showing decreased scores (red or yellow where previously green).
4. Drill into specific records to understand what changed.

### Multi-Model Comparison

Compare different models or configurations:

1. Run the same testset with different models.
2. Select all model runs in the Analysis page.
3. Compare score distributions to identify the best performer.
4. Examine specific outputs to understand quality differences.

### Metric Validation

Verify that your metrics are working correctly:

1. Select runs with known good and bad outputs.
2. Choose the metrics you want to validate.
3. Verify that scores align with expected quality assessments.
4. Adjust metric configurations if needed.

### System Evolution Tracking

Monitor how your system improves over time:

1. Select runs from different development phases.
2. Focus on your primary success metrics.
3. Observe the trend in score distributions.
4. Document improvements in run notes.

<Tip>
  Use the URL sharing feature to save specific analysis configurations. The URL automatically includes selected run IDs, making it easy to return to or share specific comparisons.
</Tip>

## Related Features

<CardGroup cols={2}>
  <Card title="A/B Comparison" href="/features/a-b-comparison" icon="split">
    For focused two-run comparisons with statistical analysis
  </Card>

  <Card title="Runs & Results" href="/features/runs" icon="play">
    Learn more about creating and managing individual runs
  </Card>
</CardGroup>