🧪 Prompt Management & Testing

Agenta

Open Source

Open-source platform for prompt management, evaluation and observability.

Arize Phoenix

Open Source

Open-source tracing, evaluation and prompt playground for LLM apps.

Braintrust

Freemium

Prompt management combined with evaluation and tracing.

DeepEval

Open Source

Open-source framework for unit-testing LLM outputs.

Google AI Studio

Free

Google’s free playground for writing and testing prompts with Gemini.

Helicone

Open Source

Open-source LLM observability with prompt versioning and cost tracking.

Langfuse

Open Source

Open-source LLM engineering platform with prompt management, tracing and evals.

LangSmith

Freemium

LangChain’s platform for tracing, evaluating and managing prompts.

Latitude

Open Source

Open-source platform for prompt engineering and AI agents.

Lunary

Open Source

Open-source monitoring and prompt management for AI apps.

Maxim AI

Freemium

End-to-end platform for prompt experimentation, evaluation and observability.

MLflow Prompt Registry

Open Source

Open-source MLOps platform with GenAI tracing, evaluation and a prompt registry.

Opik

Open Source

Open-source LLM evaluation and observability by Comet.

Orq.ai

Paid

Collaboration platform for building, testing and running LLM apps.

Pezzo

Open Source

Developer-first open-source LLMOps platform for prompt delivery.

Portkey

Freemium

AI gateway with prompt management, observability and guardrails.

Prompt flow

Open Source

Microsoft’s open-source toolkit for building and evaluating LLM flows.

Prompt Mixer

Freemium

Desktop app for creating, testing and versioning prompts.

Promptfoo

Open Source

Open-source CLI for testing prompts and red-teaming LLM apps.

PromptHub

Freemium

Git-style prompt versioning, testing and sharing for teams.

PromptLayer

Freemium

Prompt registry, versioning and request logging for LLM apps.

Promptmetheus

Freemium

Prompt engineering IDE for composing, testing and optimising prompts.

Vellum

Paid

Platform for building, testing and deploying LLM workflows and prompts.

W&B Weave

Freemium

Weights & Biases toolkit for tracing and evaluating LLM apps.