Azure AI Agent Evaluation

Version updated for https://github.com/microsoft/ai-agent-evals to version v3-beta.

• This publisher is shown as ‘verified’ by GitHub.
• This action is used across all versions by ? repositories.

Go to the GitHub Marketplace to find the latest changes.

Action Summary

The Microsoft Foundry Evaluation GitHub Action automates the offline evaluation of Microsoft Foundry Agents within CI/CD pipelines. It streamlines pre-production testing by running agents against test queries, collecting performance metrics such as latency and token usage, and generating statistical evaluation reports. This action helps identify potential issues, assess quality, and ensure meaningful improvements before deploying updates to production.

Release notes

The new release of AI Agent Evaluation GitHub Action introduces support for the new Foundry agents platform and streamlines agent evaluation in your CI/CD pipelines with access to all evaluators available in your project’s evaluator catalog.

• microsoft
• GitHub Actions