Ragas

Where it runs

• CLI
• API

Tags

• #eval
• #rag
• #llm-as-judge
• #open-source
• #ci

• View DeepEval details

  EvalFREEMIUMOpen core

  DE

  DeepEval

  Confident AI

  Pytest-style LLM evaluation framework. Open source.

  Open-source (Apache 2.0) framework for evaluating LLM apps the way Pytest tests code — assertions backed by 50+ ready metrics spanning LLM-as-judge, RAG, agents, conversation, and safety. Plugs into LangChain, CrewAI, OpenAI Agents and more. Confident AI is the paid cloud platform that adds test management, dashboards, and observability on top.

  AI insight: Modeled on Pytest — you write LLM evals as unit-test assertions and run them in CI, with 50+ metrics spanning RAG, agents, and safety.

  • eval
  • open-source
  • llm-as-judge
  • rag
  • +1

  Open
• View Patronus AI details

  EvalFREEMIUM

  PA

  Patronus AI

  Patronus AI

  Automated evaluation, guardrails, and monitoring for AI systems.

  Platform for evaluating, guarding, and monitoring LLM and agent applications across the deployment lifecycle. Anchored by research-backed evaluator models — Lynx (hallucination detection), GLIDER (LLM judge), and Percival (agent-trace debugger). Offers a self-serve API with free credits, usage-based pricing, and enterprise plans.

  AI insight: Ships proprietary evaluators — Lynx for hallucination, GLIDER as judge, Percival for agent-trace debugging — beyond prompt-based scoring.

  • eval
  • guardrails
  • monitoring
  • hallucination
  • +1

  Open
• View Braintrust details

  EvalFREEMIUM

  BR

  Braintrust

  Braintrust

  Hosted eval + tracing platform for LLM apps.

  Production-grade eval orchestration with a dashboard, dataset versioning, and OpenTelemetry tracing. Useful once eval volume outgrows a CI YAML file.

  AI insight: Where teams graduate when a CI eval file stops scaling — it adds dataset versioning and OpenTelemetry traces to the loop.

  • eval
  • tracing
  • datasets
  • production

  Open
• View Promptfoo details

  EvalFREEOSS

  PR

  Promptfoo

  Promptfoo

  Open-source LLM eval CLI. Rubric scoring + golden sets.

  YAML-driven eval harness. Pair a prompt with a goldset, define rubrics, run across multiple models in CI. Strong for catching prompt regressions before they hit production.

  AI insight: Define evals in plain YAML and run one goldset across models in CI — a prompt regression fails the build like any other test.

  • eval
  • ci
  • rubric
  • open-source

  Open