PandaProbe is an open-source agent engineering platform released by Chirpz AI in April 2026. It is an observability tool that captures and visualizes the entire lifecycle of AI agents across three layers: session, trace, and span. When agents leveraging LLMs like GPT or Claude create complex trajectories through tool calls, sub-agent branching, and user interactions, PandaProbe enables users to track, evaluate, and monitor the entire trajectory from a single dashboard. The frontend is built with Next.js, the backend with FastAPI, asynchronous tasks are handled by Celery + Redis, and data is stored in PostgreSQL 16, forming a microservices architecture. Existing LLM observability tools focus on measuring token counts and latency for single call units, failing to capture trajectory-level failures such as looping, inconsistent tool usage, and miscoordination between users and agents in multi-turn agents. To address this, PandaProbe incorporates uncertainty detection metrics proposed in its research paper, TRACER (Trajectory Risk Aggregation for Critical Episodes in Agentic Reasoning, arXiv:2602.11409), into the platform. TRACER combines multiple signals, including content-aware surprisal, situational-awareness signals, semantic/lexical repetition tracking, and tool consistency evaluation, using a tail-focused risk aggregation approach, achieving up to 37.1% and 55% improvement in AUROC and AUARC, respectively, compared to existing methods on the τ²-bench benchmark. Similar to a multi-sensor monitoring system that simultaneously monitors ECG, blood pressure, and oxygen saturation to detect critical episodes early, rather than a simple heart rate monitor, PandaProbe comprehensively diagnoses the "health status" of an agent across its entire trajectory. From a biotechnology researcher's perspective, PandaProbe is useful for quality control in automated experimental agent pipelines. For example, when a multi-omics analysis agent built with LangGraph calls more than 10 tools, from data preprocessing to statistical analysis, visualization, and report generation, PandaProbe records the LLM's decision-making process and tool call results for each step on a session basis, and scheduled evaluations can automatically detect quality regressions in production traffic. It integrates with major frameworks such as LangGraph, CrewAI, Claude Agent SDK, OpenAI Agents SDK, and Google ADK with one-line instrumentation, and supports both self-hosting (Docker Compose) and managed cloud (starting with a free Hobby tier of 100 traces per month), allowing it to be deployed in research environments that handle sensitive data while maintaining data sovereignty.

Open-source engineering platform for managing the AI agent lifecycle.

🔬 Multi-Step Experiment Agent Quality Control: When a LangGraph-based analysis agent calls tools for 10 or more steps, record the entire trajectory per session and automatically detect looping and tool misuse using TRACER metrics. Setting up scheduled evaluations with a daily cron job allows for immediate notification of quality regressions in production traffic, enabling quantitative comparison of performance changes between agent versions.