Skill Lint
Version updated for https://github.com/nkootstra/skill-lint to version v1.0.0.
- This action is used across all versions by ? repositories.
Action Type
This is a Node action using Node version 20.
Go to the GitHub Marketplace to find the latest changes.
Action Summary
Skill Lint is a GitHub Action designed to automate the evaluation, benchmarking, and improvement of AI agent skills defined in skill files within pull requests. It performs a comprehensive analysis pipeline—including linting, evaluation, benchmarking, A/B comparison, and improvement suggestions—and provides detailed feedback as PR comments and check statuses. This action streamlines the process of testing, validating, and optimizing AI agent skills, ensuring high-quality performance and efficiency.
Release notes
🎉 Initial Release
Skill Lint v1.0.0 is the first stable release of the GitHub Action that evaluates, benchmarks, and refines agent skills on pull requests.
✨ Features
Comprehensive Skill Evaluation Pipeline
- Lint — Static analysis of skill structure (name, description, routing table, orphaned references)
- Eval — Run test cases against the LLM with hard constraints and LLM-as-judge scoring
- Benchmark — Compute pass rate, token usage, and latency metrics
- A/B Compare — Compare head vs base branch performance to detect regressions
- Suggest — Auto-generate improvement recommendations for skills
Multi-Provider Support
- Anthropic — Native support with Sonnet, Opus, and other Claude models
- OpenAI — Full support for GPT-4o, GPT-4, and other OpenAI models
- Claude Code — Direct integration with Claude GitHub App (Max/Pro subscription)
- LiteLLM — Custom proxy support for any model or on-premise deployment
GitHub Integration
- Posts detailed evaluation results as PR comments
- Sets check status for pass/fail with breakdown by skill
- Supports multiple skills per PR
- Non-blocking evaluations (can be made advisory via config)
Marketplace Ready
- Published to GitHub Marketplace
- Ready for one-click installation in any repository
- Configurable via
.skill-lint.yml - Works out-of-the-box with sensible defaults
📦 What’s Included
- Action runtime — Node.js 20 bundled with all dependencies
- Evaluation engine — Full linting, testing, and benchmarking pipeline
- CLI & configuration —
.skill-lint.ymlconfiguration support - Documentation — Installation, provider setup, evals guide, custom rubrics, Marketplace publishing
- Example workflows — CI and release workflows for Marketplace updates
🚀 Quick Start
Add to your .github/workflows/skill-lint.yml:
name: Skill Lint
on:
pull_request:
paths:
- "skills/**"
- ".skill-lint.yml"
permissions:
contents: read
pull-requests: write
checks: write
jobs:
lint-skills:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: nkootstra/skill-lint@v1
with:
provider: anthropic
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
📋 Requirements
- Anthropic, OpenAI, or compatible LLM API key
- Skills in standard directory structure with
SKILL.mdand optionalSKILL.eval.ymlfiles .skill-lint.ymlconfiguration file (optional, uses defaults if absent)
📚 Documentation
🏭 In This Release
- Full evaluation pipeline implementation
- Multi-provider LLM support (Anthropic, OpenAI, Claude Code, LiteLLM)
- GitHub PR comment and check integration
- CLI for local skill evaluation
- Comprehensive documentation
- CI/CD workflows for automated testing and Marketplace releases
- Pre-bundled dist/ for immediate GitHub Marketplace availability
License: MIT
Maintainer: @nkootstra