EvalView - AI Agent Testing

Version updated for https://github.com/hidai25/eval-view to version v0.1.3.

  • This action is used across all versions by ? repositories.

Go to the GitHub Marketplace to find the latest changes.

Release notes

EvalView GitHub Action

Pytest-style testing framework for AI agents — now available as a GitHub Action.

Usage

- uses: hidai25/eval-view@v0.1.3
  with:
    openai-api-key: ${{ secrets.OPENAI_API_KEY }}

Features

- 🧪 Test LangGraph, CrewAI, OpenAI, Anthropic, and custom agents
- ⚡ Parallel test execution (4 workers by default)
- 📊 Auto-generated HTML reports
- 💬 PR comments with test results
- 🤖 LLM-as-judge output evaluation
- 💰 Cost and latency threshold checks

Action Inputs

| Input          | Description                     | Default               |
|----------------|---------------------------------|-----------------------|
| openai-api-key | OpenAI API key for LLM-as-judge | -                     |
| config-path    | Path to config file             | .evalview/config.yaml |
| max-workers    | Parallel workers                | 4                     |
| fail-on-error  | Fail on test failure            | true                  |

Full Documentation

See https://github.com/hidai25/eval-view#github-action-recommended for complete usage 
examples.