Skip to content

Implement cost and latency tracking #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ArtyomVancyan merged 9 commits into from
Mar 2, 2026
Merged

Implement cost and latency tracking #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
85780ce
Enhance Formatter class with metrics extraction and improved message …
ArtyomVancyan Mar 1, 2026
ddb6585
Add metrics extraction to run status reporting in streamer
ArtyomVancyan Mar 1, 2026
4a5bec7
Add script for model metadata extraction
ArtyomVancyan Mar 2, 2026
f238f1c
Fetch the latest models metadata
ArtyomVancyan Mar 2, 2026
c966d41
Implement metrics calculation and formatting
ArtyomVancyan Mar 2, 2026
c172cba
Implement metrics bar inside the inspect panel
ArtyomVancyan Mar 2, 2026
585924c
Remove langsmith dependency from project configuration
ArtyomVancyan Mar 2, 2026
43d943a
Merge branch 'main' into cost-and-latency-tracking
ArtyomVancyan Mar 2, 2026
0859519
Enhance feature comparison table and adjust demo image size
ArtyomVancyan Mar 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 17 additions & 21 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ especially useful when working with large networks: graphs with many nodes, bran
reason about from the logs alone.

<p align="center">
<img alt="Demo" src="https://github.com/user-attachments/assets/1db519fb-0dd9-4fee-8bc8-f6b12cbf1342" width="80%">
<img alt="Demo" src="https://github.com/user-attachments/assets/1db519fb-0dd9-4fee-8bc8-f6b12cbf1342" width="800px">
</p>

## Why it helps
Expand Down Expand Up @@ -43,26 +43,22 @@ it in while you're actively building - it has no effect on how the agent behaves

## Features

| Feature | [LangGraphics](https://github.com/proactive-agent/langgraphics) | [LangFuse](https://github.com/langfuse/langfuse) | [LangSmith Studio](https://smith.langchain.com) |
|------------------------------------|-----------------------------------------------------------------|--------------------------------------------------|-------------------------------------------------|
| Open-source | ✅ | ✅ | ❌ |
| Unlimited free usage | ✅ | ✅ | ❌ |
| Self-hosting supported | ✅ | ✅ | ❌ |
| No vendor lock-in | ✅ | ✅ | ❌ |
| Works without external services | ✅ | ❌ | ❌ |
| Simple setup | ✅ | ❌ | ❌ |
| One-line integration | ✅ | ❌ | ❌ |
| No API key required | ✅ | ❌ | ❌ |
| No instrumentation required | ✅ | ❌ | ❌ |
| Runs fully locally | ✅ | ❌ | ❌ |
| Native LangGraph visualization | ✅ | ❌ | ❌ |
| Real-time execution graph | ✅ | ❌ | ❌ |
| Data stays local by default | ✅ | ❌ | ❌ |
| Low learning curve | ✅ | ❌ | ❌ |
| Built-in prompt evaluation | ❌ | ❌ | ✅ |
| Built-in observability dashboards | ❌ | ✅ | ✅ |
| Built-in cost and latency tracking | ❌ | ✅ | ✅ |
| Production monitoring capabilities | ❌ | ✅ | ✅ |
| Feature | [LangGraphics](https://github.com/proactive-agent/langgraphics) | [LangFuse](https://github.com/langfuse/langfuse) | [LangSmith](https://smith.langchain.com) |
|-------------------------|----------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|
| Fully local | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| Standalone | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| Easy to learn | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| One-line setup | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| Data stays local | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| No API key required | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| Live execution graph | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| No refactoring required | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| Self-hosted | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| No vendor lock-in | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| Unlimited free usage | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) |
| Graph visualization | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) |
| Cost & latency tracking | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) |
| Prompt evaluation | ![🟥](https://github.com/user-attachments/assets/ebe12afc-ae2e-42b1-a058-e93353ff87c4) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) | ![✅](https://github.com/user-attachments/assets/b3c25b41-567c-4c26-bc02-ee3e40fd57c1) |

## Contribute

Expand Down
14 changes: 9 additions & 5 deletions langgraphics-web/src/components/GraphCanvas.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import {type ReactNode, useCallback, useEffect, useState} from "react";
import {useCallback, useEffect, useState} from "react";
import {Background, type ColorMode, type Edge, type Node, type NodeTypes, ReactFlow} from "@xyflow/react";
import {Controls} from "./Controls";
import {CustomNode} from "./CustomNode";
import {useFocus} from "../hooks/useFocus";
import type {EdgeData, ExecutionEvent, InspectorMode, NodeData, ViewMode} from "../types";
import {InspectPanel} from "./InspectPanel";
import type {EdgeData, ExecutionEvent, InspectorMode, NodeData, NodeEntry, ViewMode} from "../types";
import type {RankDir} from "../layout";

const nodeTypes: NodeTypes = {custom: CustomNode as NodeTypes[string]};
Expand All @@ -12,16 +13,16 @@ interface GraphCanvasProps {
nodes: Node<NodeData>[];
edges: Edge<EdgeData>[];
events: ExecutionEvent[];
nodeEntries: NodeEntry[];
activeNodeIds: string[];
inspect: ReactNode;
initialMode: ViewMode;
initialRankDir?: RankDir;
initialColorMode?: ColorMode;
initialInspect?: InspectorMode;
onRankDirChange?: (v: RankDir) => void;
}

export function GraphCanvas({nodes, edges, events, activeNodeIds, inspect, initialMode = "auto", initialInspect = "off", initialColorMode = "system", initialRankDir = "TB", onRankDirChange}: GraphCanvasProps) {
export function GraphCanvas({nodes, edges, events, activeNodeIds, nodeEntries, initialMode = "auto", initialInspect = "off", initialColorMode = "system", initialRankDir = "TB", onRankDirChange}: GraphCanvasProps) {
const [rankDir, setRankDir] = useState<RankDir>(initialRankDir);
const [colorMode, setColorMode] = useState<ColorMode>(initialColorMode);
const [inspectorMode, setInspectorMode] = useState<InspectorMode>(initialInspect);
Expand Down Expand Up @@ -65,7 +66,10 @@ export function GraphCanvas({nodes, edges, events, activeNodeIds, inspect, initi
/>
<Background/>
<div className={`inspect-wrapper-${inspectorMode}`}>
{inspect}
<InspectPanel
colorMode={colorMode}
nodeEntries={nodeEntries}
/>
</div>
</ReactFlow>
);
Expand Down
10 changes: 9 additions & 1 deletion langgraphics-web/src/components/InspectPanel.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
import Tree from "antd/es/tree";
import type {TreeDataNode} from "antd";
import ReactMarkdown from "react-markdown";
import type {ColorMode} from "@xyflow/react";
import {useCallback, useEffect, useMemo, useState} from "react";
import type {NodeEntry} from "../types";
import {Metrics} from "./Metrics";

export function InspectPanel({nodeEntries}: { nodeEntries: NodeEntry[] }) {
export function InspectPanel({colorMode, nodeEntries}: { colorMode: ColorMode, nodeEntries: NodeEntry[] }) {
const [selectedKey, setSelectedKey] = useState<string>("");

const expandedKeys = useMemo(() => {
Expand Down Expand Up @@ -100,6 +102,12 @@ export function InspectPanel({nodeEntries}: { nodeEntries: NodeEntry[] }) {
<div className="inspect-detail-pane">
{selectedEntry && (
<>
{selectedEntry.metrics && (
<Metrics
colorMode={colorMode}
metrics={selectedEntry.metrics}
/>
)}
{system && (
<div className="inspect-detail-section">
<span className={`inspect-section-label ${system.role ?? ""}`}>
Expand Down
Loading
Loading