openjudge
Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.
Packaged view
This page reorganizes the original catalog entry around fit, installability, and workflow context first. The original raw source lives below.
Install command
npx @skill-hub/cli install agentscope-ai-openjudge-openjudge
Repository
Skill path: skills/openjudge
Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.
Open repositoryBest for
Primary workflow: Design Product.
Technical facets: Full Stack, Data / AI, Designer.
Target audience: Development teams looking for install-ready agent workflows..
License: Unknown.
Original source
Catalog source: SkillHub Club.
Repository owner: agentscope-ai.
This is still a mirrored public skill entry. Review the repository before installing into production workflows.
What it helps with
- Install openjudge into Claude Code, Codex CLI, Gemini CLI, or OpenCode workflows
- Review https://github.com/agentscope-ai/OpenJudge before adding openjudge to shared team environments
- Use openjudge for development workflows
Works across
Favorites: 0.
Sub-skills: 0.
Aggregator: No.