The DevEx Compass

Ergonomics runs west to east, production parity south to north. A public checklist places every dot.

How a dot is placed

No dot is placed by hand. Each tool earns 10 points per criterion it meets, with a source. Ties sit in a small cluster around the same point.

Developer ergonomics west → east

  1. One command starts it. A single command brings up the whole thing it manages.
  2. Works with little config. Useful out of the box, or with a short generated config.
  3. Fast feedback on change. Live reload, file sync or incremental rebuilds in seconds.
  4. Clear errors and diagnostics. Says what broke and how to fix it, or ships a doctor-style check.
  5. Built-in UI. A web dashboard, TUI or desktop app, not only raw logs.
  6. Quickstart in minutes. Official docs get a newcomer to a working result in about ten minutes.
  7. macOS, Linux and Windows. Supported on all three, natively or through WSL.
  8. Local, no account. Runs on your machine without signing up or a network round trip.
  9. Light and quick to start. Starts in seconds and does not need a large VM or many GB of RAM.
  10. Agent-ready. An MCP server, a JSON output mode or an API an AI agent can drive.

Complexity and production parity south → north

  1. Real containers. Runs or builds OCI containers.
  2. Many services at once. Brings up and coordinates several services together.
  3. Prod-like networking. Service-to-service networking, DNS or ingress like production.
  4. Real stateful dependencies. Databases, queues or volumes, not only in-memory fakes.
  5. Same artifacts as production. Uses the images, bundles or lockfiles that production uses.
  6. Kubernetes semantics. Manifests, pods, Helm charts or a real Kubernetes API.
  7. Cloud APIs. Emulates or reaches real cloud provider APIs.
  8. Prod-like config and secrets. Environment, config and secrets handled the way production handles them.
  9. Logs, metrics and traces. Exposes telemetry the way production does.
  10. Scales to big teams. Built for many services and many developers sharing a setup.
The same tools as a table
ToolFamilyCategoryRingLicenseErgonomicsParity
LangGraph Stateful graphs for LLM agents AI & agents Agent frameworks Adopt Open source 40 20
LangChain Model, tool and retriever building blocks for LLM apps AI & agents Agent frameworks Adopt Open source 40 0
CrewAI Role-based multi-agent crews and flows in Python AI & agents Agent frameworks Adopt Open source 60 0
AutoGen Microsoft's multi-agent conversation framework AI & agents Agent frameworks Hold Open source 40 10
AG2 Community fork of AutoGen 0.2 for multi-agent apps AI & agents Agent frameworks Adopt Open source 40 10
Microsoft Agent Framework Successor to Semantic Kernel and AutoGen for .NET and Python AI & agents Agent frameworks Adopt Open source 50 20
Semantic Kernel Microsoft SDK for AI plugins in C#, Python and Java AI & agents Agent frameworks Adopt Open source 40 10
Mastra TypeScript framework for agents, workflows and RAG AI & agents Agent frameworks Adopt Open source 90 20
OpenAI Agents SDK Lightweight agents, handoffs and guardrails from OpenAI AI & agents Agent frameworks Trial Open source 40 10
Claude Agent SDK The Claude Code agent loop as a library AI & agents Agent frameworks Trial Open source 50 0
Google ADK Google's Agent Development Kit for Python, Java, Go and TS AI & agents Agent frameworks Adopt Open source 50 20
Pydantic AI Type-safe Python agent framework from the Pydantic team AI & agents Agent frameworks Adopt Open source 60 10
LlamaIndex Data and retrieval framework for LLM agents AI & agents Agent frameworks Trial Open source 30 10
Haystack Pipelines for RAG and agents from deepset AI & agents Agent frameworks Adopt Open source 40 20
DSPy Program and optimize LM pipelines instead of prompting AI & agents Agent frameworks Adopt Open source 40 0
Vercel AI SDK TypeScript toolkit for LLM apps, streaming UI and agents AI & agents Agent frameworks Adopt Open source 60 10
smolagents Hugging Face's minimal code-writing agents AI & agents Agent frameworks Adopt Open source 50 10
Agno Python agents and teams with a runtime and control plane AI & agents Agent frameworks Adopt Open source 50 20
Letta Stateful agents with long-term memory, formerly MemGPT AI & agents Agent frameworks Trial Open source 50 20
Temporal Durable execution for long-running agent workflows AI & agents Agent frameworks Adopt Open source 60 70
Inngest Durable steps, queues and AgentKit for serverless code AI & agents Agent frameworks Adopt Free tier, paid plans 80 20
Restate Durable execution engine for agents and services AI & agents Agent frameworks Adopt Open source 60 50
Model Context Protocol Open protocol and SDKs connecting agents to tools and data AI & agents Agent tools and MCP Adopt Open source 50 0
MCP Inspector Visual test and debug tool for MCP servers AI & agents Agent tools and MCP Adopt Open source 90 0
Docker MCP Toolkit Run catalog MCP servers as containers behind one gateway AI & agents Agent tools and MCP Trial Free tier, paid plans 50 30
Smithery Registry and hosting for MCP servers AI & agents Agent tools and MCP Hold Free tier, paid plans 40 0
MCP Registry Official community registry of public MCP servers AI & agents Agent tools and MCP Trial Open source 20 0
FastMCP Pythonic framework for MCP servers and clients AI & agents Agent tools and MCP Adopt Open source 80 0
Composio Hosted tool integrations and auth for agents AI & agents Agent tools and MCP Trial Free tier, paid plans 40 10
skills.sh Directory and CLI for installing agent skills AI & agents Agent tools and MCP Assess Open source 50 0
Stagehand Browser automation SDK for agents, by Browserbase AI & agents Agent tools and MCP Adopt Open source 50 0
Playwright MCP MCP server that drives a browser through Playwright AI & agents Agent tools and MCP Trial Open source 60 0
E2B Cloud sandboxes for running agent-generated code AI & agents Agent tools and MCP Adopt Free tier, paid plans 50 20
Claude Code Anthropic's agentic coding tool for the terminal and IDEs AI & agents Coding agents Adopt Paid 90 0
Codex OpenAI's coding agent for terminal, IDE and cloud AI & agents Coding agents Trial Open source 80 0
Cursor AI-first code editor forked from VS Code AI & agents Coding agents Adopt Free tier, paid plans 60 0
Gemini CLI Google's open-source terminal agent for Gemini AI & agents Coding agents Trial Open source 80 0
GitHub Copilot GitHub's AI pair programmer, agent mode and cloud agent AI & agents Coding agents Adopt Free tier, paid plans 60 0
Devin Desktop (formerly Windsurf) Cognition's agentic IDE, renamed from Windsurf AI & agents Coding agents Trial Free tier, paid plans 60 0
Cline Open-source coding agent for VS Code and the terminal AI & agents Coding agents Adopt Open source 70 0
Roo Code VS Code agent forked from Cline, now discontinued AI & agents Coding agents Hold Open source 50 0
Aider Git-aware AI pair programming in the terminal AI & agents Coding agents Trial Open source 60 0
Continue Open-source IDE agent and CI checks with any model AI & agents Coding agents Adopt Open source 60 0
Zed Fast Rust editor with built-in agent panel and ACP AI & agents Coding agents Adopt Open source 80 0
Warp Agentic terminal and development environment AI & agents Coding agents Adopt Free tier, paid plans 50 0
Amp Coding agent for terminal and editors, from Sourcegraph AI & agents Coding agents Trial Free tier, paid plans 50 0
OpenCode Open-source terminal coding agent for any model AI & agents Coding agents Adopt Open source 90 0
goose Local, extensible open-source AI agent built on MCP AI & agents Coding agents Adopt Open source 60 0
Kiro AWS's spec-driven agentic IDE and CLI AI & agents Coding agents Trial Free tier, paid plans 50 0
Jules Google's asynchronous cloud coding agent AI & agents Coding agents Trial Free tier, paid plans 30 10
Devin Cognition's autonomous cloud software engineer AI & agents Coding agents Adopt Paid 40 20
Paradev (formerly Superset) Desktop app to run many coding agents in parallel worktrees AI & agents Coding agents Trial Free tier, paid plans 40 0
Conductor Mac app to run parallel Claude Code and Codex agents AI & agents Coding agents Trial Free, closed source 40 0
Ollama Run open models locally behind a simple API AI & agents Local models Trial Open source 70 10
LM Studio Desktop app to download and serve local models AI & agents Local models Trial Free, closed source 70 0
llama.cpp C/C++ inference for GGUF models on CPU and GPU AI & agents Local models Trial Open source 50 10
vLLM High-throughput LLM serving engine AI & agents Local models Trial Open source 30 40
LocalAI Self-hosted OpenAI-compatible API for many model types AI & agents Local models Adopt Open source 60 20
Docker Model Runner Pull and run models as OCI artifacts with Docker AI & agents Local models Adopt Open source 60 30
Jan Open-source desktop ChatGPT alternative that runs offline AI & agents Local models Trial Open source 70 0
MLX LM Run and fine-tune LLMs on Apple silicon with MLX AI & agents Local models Trial Open source 40 0
Langfuse Open-source LLM tracing, evals and prompt management AI & agents LLM observability Adopt Open source 60 60
LangSmith LangChain's tracing, evals and deployment platform AI & agents LLM observability Adopt Free tier, paid plans 50 20
Arize Phoenix Open-source LLM tracing and evaluation on OpenTelemetry AI & agents LLM observability Adopt Open source 80 30
Helicone Proxy-based LLM logging and AI gateway AI & agents LLM observability Hold Open source 30 10
OpenLLMetry OpenTelemetry instrumentation for LLM libraries AI & agents LLM observability Trial Open source 50 10
Braintrust Evals, logging and prompt playground for AI products AI & agents LLM observability Adopt Free tier, paid plans 40 20
promptfoo CLI for LLM evals and red teaming in CI AI & agents LLM observability Trial Open source 70 0
Opik Open-source LLM tracing and evals from Comet AI & agents LLM observability Adopt Open source 60 50