Patronus AI

Where it runs

• Web
• API

Tags

• #eval
• #guardrails
• #monitoring
• #hallucination
• #agents

• View DeepEval details

  EvalFREEMIUMOpen core

  DE

  DeepEval

  Confident AI

  Pytest-style LLM evaluation framework. Open source.

  Open-source (Apache 2.0) framework for evaluating LLM apps the way Pytest tests code — assertions backed by 50+ ready metrics spanning LLM-as-judge, RAG, agents, conversation, and safety. Plugs into LangChain, CrewAI, OpenAI Agents and more. Confident AI is the paid cloud platform that adds test management, dashboards, and observability on top.

  AI insight: Modeled on Pytest — you write LLM evals as unit-test assertions and run them in CI, with 50+ metrics spanning RAG, agents, and safety.

  • eval
  • open-source
  • llm-as-judge
  • rag
  • +1

  Open
• View Ragas details

  EvalFREEOSS

  RA

  Ragas

  Exploding Gradients

  Open-source evaluation toolkit for RAG and LLM applications.

  Open-source (Apache-2.0) Python framework for evaluating retrieval-augmented generation and LLM apps. Provides reference-free metrics — faithfulness, answer relevancy, context precision/recall — plus knowledge-graph-based synthetic test generation. Integrates with LangChain, LlamaIndex, and CI pipelines.

  AI insight: Popularized reference-free RAG metrics — faithfulness, context precision — scored by an LLM judge, so you evaluate without gold answers.

  • eval
  • rag
  • llm-as-judge
  • open-source
  • +1

  Open
• View Braintrust details

  EvalFREEMIUM

  BR

  Braintrust

  Braintrust

  Hosted eval + tracing platform for LLM apps.

  Production-grade eval orchestration with a dashboard, dataset versioning, and OpenTelemetry tracing. Useful once eval volume outgrows a CI YAML file.

  AI insight: Where teams graduate when a CI eval file stops scaling — it adds dataset versioning and OpenTelemetry traces to the loop.

  • eval
  • tracing
  • datasets
  • production

  Open
• View Promptfoo details

  EvalFREEOSS

  PR

  Promptfoo

  Promptfoo

  Open-source LLM eval CLI. Rubric scoring + golden sets.

  YAML-driven eval harness. Pair a prompt with a goldset, define rubrics, run across multiple models in CI. Strong for catching prompt regressions before they hit production.

  AI insight: Define evals in plain YAML and run one goldset across models in CI — a prompt regression fails the build like any other test.

  • eval
  • ci
  • rubric
  • open-source

  Open