DeepEval

Open-source framework for unit-testing LLM outputs.

Open Source Prompt Management & Testing github.com

About DeepEval

DeepEval lets you write tests for LLM outputs with ready-made metrics, run them in CI, and compare prompt versions. The Confident AI platform adds collaboration and regression tracking.

Feels like pytest for prompts.

FAQ

What is DeepEval?

Open-source framework for unit-testing LLM outputs.

Is DeepEval free?

Yes. DeepEval is open source and free to use or self-host.

What are some alternatives to DeepEval?

Agenta, Arize Phoenix, Braintrust, Google AI Studio, Helicone.

Alternatives to DeepEval

All prompt management & testing →

Agenta

Open Source

Open-source platform for prompt management, evaluation and observability.

Visit ↗

Arize Phoenix

Open Source

Open-source tracing, evaluation and prompt playground for LLM apps.

Visit ↗

Braintrust

Freemium

Prompt management combined with evaluation and tracing.

Visit ↗

Google AI Studio

Free

Google’s free playground for writing and testing prompts with Gemini.

Visit ↗

Helicone

Open Source

Open-source LLM observability with prompt versioning and cost tracking.

Visit ↗

Langfuse

Open Source

Open-source LLM engineering platform with prompt management, tracing and evals.

Visit ↗

LangSmith

Freemium

LangChain’s platform for tracing, evaluating and managing prompts.

Visit ↗

Latitude

Open Source

Open-source platform for prompt engineering and AI agents.

Visit ↗

Lunary

Open Source

Open-source monitoring and prompt management for AI apps.

Visit ↗

Maxim AI

Freemium

End-to-end platform for prompt experimentation, evaluation and observability.

Visit ↗

MLflow Prompt Registry

Open Source

Open-source MLOps platform with GenAI tracing, evaluation and a prompt registry.

Visit ↗

Opik

Open Source

Open-source LLM evaluation and observability by Comet.

Visit ↗