🛠️Tool Registry

Track the true cost of AI agents by metering any tool, API, or service—not just tokens.

The Tool Registry enables organizations to track costs beyond AI tokens. While most AI cost tracking focuses on LLM tokens, real-world agentic solutions incur many other costs: external API calls, third-party services, and even human time. The Tool Registry captures these costs so you can calculate true ROI for your AI investments.

Token costs are typically the smallest part of what an AI agent costs to run. In the example below, tool costs are 10x the token costs—a $9.68 tool bill alongside $0.95 in tokens, for a total of $10.63. External API calls, data services, and other tool invocations are often the dominant cost in production agentic workflows, yet they're invisible to LLM gateways that only see what flows through the model provider.

---

Why Tool Registry?

AI agents don't just consume tokens—they make decisions that cost real money:

• External API Calls: Credit report lookups ($25/call), mapping services, payment processors

• Third-Party Services: Web scraping services, data enrichment APIs, document processing

• MCP Servers: Model Context Protocol servers that extend agent capabilities

• Human-in-the-Loop: Human review steps that cost time and money

Without tracking these costs, you only see part of the picture. An agent that appears cost-efficient based on token usage might actually be expensive when you factor in the $25 credit check it triggers for every customer interaction—a cost that never appears in your LLM provider bill.

The Tool Registry solves this by letting you define any cost source—then meter usage against it automatically.

---

Use Cases

External API Cost Tracking

Your customer service agent pulls credit reports from Experian at $25 per lookup. Register the API as a tool and track every invocation:

• Tool ID: experian-credit-check

• Tool Type: REST API

• Unit Price: $25.00 per request

Now every credit check is attributed to the agent, customer, and workflow that triggered it.

Human Time Allocation

Your compliance workflow requires human review for flagged transactions. Track the cost of human involvement:

• Tool ID: compliance-review

• Tool Type: Custom

• Unit Price: $1.50 per minute (based on $90/hour reviewer cost)

Meter the duration of each review to understand the true cost of human-in-the-loop workflows.

MCP Server Costs

Your agents use MCP servers for GitHub operations, web searches, and database queries. Each server has different cost profiles:

• Tool ID: mcp-github

• Tool Type: MCP Server

• Unit Price: $0.001 per operation

Track MCP server usage alongside token costs for complete visibility.

Tiered Pricing for Volume Discounts

Some services offer volume discounts. Configure tiered pricing to reflect actual costs:

• First 1,000 requests: $0.10 each

• 1,001–10,000 requests: $0.08 each

• 10,001+ requests: $0.05 each

The Tool Registry automatically applies the correct tier based on usage volume.

---

Creating a Tool

Navigate to Tools in the sidebar to access the Tool Registry.

Required Fields

Field	Description	Example
Tool ID	Unique identifier (letters, numbers, hyphens, underscores)	experian-credit-check
Name	Human-readable display name	Experian Credit Check API
Tool Type	Category of the tool	REST API

Tool Types

Type	Use For
SDK	SDK-based integrations
MCP Server	Model Context Protocol servers
AI Service	AI services beyond LLM completions
REST API	External REST APIs
Local Function	Local code functions with associated costs
Custom	Any other cost source

Optional Fields

Field	Description
Description	Detailed explanation of the tool's purpose
Provider	Vendor or service provider name
Enabled	Toggle to activate/deactivate metering

---

Configuring Pricing

Each tool can have one or more pricing elements. This flexibility handles complex pricing models where a single tool call might incur multiple types of charges.

Flat Rate Pricing

For tools with simple per-call pricing:

1. Click Add Element

2. Enter a name (e.g., "requests")

3. Set the Unit Price

4. Choose an Aggregation Type:

   • Sum: Add all values together

   • Count: Count occurrences

   • Average: Calculate mean

   • Maximum: Highest value

   • Distinct: Unique values only

Volume-Based Tiered Pricing

For tools with volume discounts:

1. Click Add Element

2. Click Add Tier to define pricing tiers

3. For each tier, specify:

   • Up To: Maximum quantity for this tier (leave blank for unlimited)

   • Unit Price: Price per unit within this tier

Example: Web Scraping Service

Tier	Up To	Unit Price
1	100	$0.05
2	1,000	$0.03
3	∞	$0.01

---

Metering Tool Events

Once tools are registered, meter usage via the Tool Event API.

API Endpoint

POST /v2/tool/events

Required Fields

Field	Description
toolId	The tool's unique identifier
timestamp	ISO 8601 timestamp of the event

Optional Fields

Field	Description
operation	Specific operation performed (e.g., "scrape", "lookup")
durationMs	Duration in milliseconds
success	Whether the call succeeded
errorMessage	Error details if failed
costUsd	Direct cost override in USD
agent	Agent that triggered the call
organizationId	Organization identifier for multi-tenant tracking
subscriberCredential	Subscriber or customer credential for billing attribution
product	Product or application name
workflowId	Workflow for multi-step tracking
traceId	Distributed tracing correlation
usageMetadata	Additional key-value data

Example: Credit Check API Call

{
  "toolId": "experian-credit-check",
  "operation": "pull-report",
  "durationMs": 2340,
  "success": true,
  "timestamp": "2025-01-15T12:00:00Z",
  "agent": "loan-processor",
  "organizationId": "org-acme-bank",
  "subscriberCredential": "sub-456",
  "traceId": "trace-abc-123",
  "usageMetadata": {
    "report_type": "full",
    "customer_id": "cust-456"
  }
}

Example: Human Review Time

{
  "toolId": "compliance-review",
  "operation": "manual-review",
  "durationMs": 180000,
  "success": true,
  "timestamp": "2025-01-15T14:30:00Z",
  "agent": "compliance-workflow",
  "usageMetadata": {
    "reviewer": "[email protected]",
    "decision": "approved"
  }
}

Example: Failed Tool Call

{
  "toolId": "github-api",
  "operation": "fetch-repo",
  "durationMs": 500,
  "success": false,
  "errorMessage": "Repository not found",
  "timestamp": "2025-01-15T12:00:00Z",
  "agent": "code-analyzer"
}

For complete API details, see the Tool Event API reference in our Platform API Documentation.

---

Viewing Tool Costs

Tool costs appear alongside token costs in your analytics, giving you a unified view of the full economic footprint of your agents.

Cost Iceberg by Agent

The Tools dashboard shows token costs and tool costs side-by-side, broken down by agent. The Tool/Token Ratio metric makes it immediately clear when tool costs dominate—which is the common case in production workflows that call external services.

In the example above: total cost $10.63, token cost $0.95, tool cost $9.68, tool/token ratio of over 10x. The four agents (code-assistant, customer-support, data-pipeline, research-analyst) each have different cost profiles—tool-heavy agents are easy to identify at a glance.

Tool Cost Over Time

The Tool Cost Over Time chart breaks down tool spend by individual tool across a time range. Use this to see which tools are driving cost spikes and how your external service usage trends over time.

Each tool is shown as a separate series. In the example: snowflake_query (29%) and postgres_query (26%) dominate, with aws_bedrock, web_scraper, firecrawl, tavily, github_api, and stripe_api also tracked. Cost spikes are immediately visible—here, two large snowflake_query calls occurred mid-afternoon.

Trace Cost Distribution

The Trace Cost Distribution scatter plot shows the cost of every individual trace, letting you identify outliers and understand your cost percentiles.

Percentile stats (p50, p90, p95, p99) and a configurable threshold line let you quickly identify traces that exceeded expected cost bounds. Each dot is a single trace, colored by agent—making it easy to see whether cost outliers are concentrated in a specific agent or distributed across your fleet.

Other Surfaces

Tool costs are also attributed using the same dimensions as AI completions:

• Provider Dashboard: See tool costs aggregated with token costs at the organization level

• Trace Analytics: View tool calls within individual traces alongside the LLM calls they accompanied

• Cost & Performance Alerts: Set alerts on tool-based cost metrics

Tool costs use the same attribution dimensions as AI completions—organization, agent, product, subscriber—so you can slice and filter the same way across all cost types.

---

Best Practices

Use Descriptive Tool IDs

Choose tool IDs that clearly identify the service and operation:

• Good: experian-credit-check, google-maps-geocode, slack-send-message

• Avoid: api1, tool, external

Track Both Success and Failure

Meter failed tool calls too—they often still incur costs. Use the success and errorMessage fields to distinguish outcomes while maintaining cost visibility.

Include Attribution Metadata

Always include agent, organizationId, subscriberCredential, and traceId when available. This enables:

• Cost breakdown by agent

• Organization and subscriber-level billing

• End-to-end trace analysis

Start with High-Cost Tools

Prioritize registering tools with significant per-call costs. A $25 credit check matters more than a $0.001 API call for initial ROI analysis.

Review Tool Costs Regularly

Use the Tool Registry alongside Budget Monitoring to set spending limits and receive alerts when tool costs exceed expectations.

---

Summary

The Tool Registry transforms Revenium from an AI token tracker into a complete cost attribution platform. By capturing every cost your agents incur—API calls, third-party services, human time—you can calculate true ROI and make informed decisions about your AI investments.

Register your high-value tools, meter their usage, and see the complete picture of what your AI solutions actually cost.