Observability

Overview

Observability Drivers are used by Observability to send telemetry (metrics and traces) related to the execution of an LLM application. The telemetry can be used to monitor the application and to diagnose and troubleshoot issues. All Observability Drivers implement the following methods:

__enter__() sets up the Driver.
__exit__() tears down the Driver.
observe() wraps all functions and methods marked with the @observable decorator. At a bare minimum, implementations call the wrapped function and return its result (a no-op). This enables the Driver to generate telemetry related to the invocation's call arguments, return values, exceptions, latency, etc.

Observability Drivers

Griptape Cloud

Info

This driver requires the drivers-observability-griptape-cloud extra.

The Griptape Cloud Observability Driver instruments @observable functions and methods with metrics and traces for use with the Griptape Cloud.

Note

For the Griptape Cloud Observability Driver to function as intended, it must be run from within either a Managed Structure on Griptape Cloud or locally via the Skatepark Emulator.

Here is an example of how to use the GriptapeCloudObservabilityDriver with the Observability context manager to send the telemetry to Griptape Cloud:

from griptape.drivers.observability.griptape_cloud import GriptapeCloudObservabilityDriver
from griptape.observability import Observability
from griptape.rules import Rule
from griptape.structures import Agent

observability_driver = GriptapeCloudObservabilityDriver()

with Observability(observability_driver=observability_driver):
    agent = Agent(rules=[Rule("Output one word")])
    agent.run("Name an animal")

OpenTelemetry

Info

This driver requires the drivers-observability-opentelemetry extra.

The OpenTelemetry Observability Driver instruments @observable functions and methods with metrics and traces for use with OpenTelemetry. You must configure a destination for the telemetry by providing a SpanProcessor to the Driver.

Here is an example of how to use the OpenTelemetryObservabilityDriver with the Observability context manager to output the telemetry directly to the console:

from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter

from griptape.drivers.observability.open_telemetry import OpenTelemetryObservabilityDriver
from griptape.observability import Observability
from griptape.rules import Rule
from griptape.structures import Agent

observability_driver = OpenTelemetryObservabilityDriver(
    service_name="name-an-animal", span_processor=BatchSpanProcessor(ConsoleSpanExporter())
)

with Observability(observability_driver=observability_driver):
    agent = Agent(rules=[Rule("Output one word")])
    agent.run("Name an animal")

Prompt Driver metrics

To export metrics as well as traces, provide an OpenTelemetry SDK MeterProvider with configured metric readers. If meter_provider is omitted, the Driver uses OpenTelemetry's global meter provider. Without a configured meter provider, no metrics are exported and existing trace-only configurations continue to work.

from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import ConsoleMetricExporter, PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter

from griptape.drivers.observability.open_telemetry import OpenTelemetryObservabilityDriver
from griptape.observability import Observability
from griptape.structures import Agent

meter_provider = MeterProvider(
    resource=Resource.create({"service.name": "name-an-animal"}),
    metric_readers=[PeriodicExportingMetricReader(ConsoleMetricExporter())],
)
try:
    observability_driver = OpenTelemetryObservabilityDriver(
        service_name="name-an-animal",
        span_processor=BatchSpanProcessor(ConsoleSpanExporter()),
        meter_provider=meter_provider,
    )
    with Observability(observability_driver=observability_driver):
        Agent().run("Name an animal")
finally:
    meter_provider.shutdown()

Each completed PromptDriver.run() records the following GenAI metrics, including streaming calls:

Metric	Instrument	Unit
`gen_ai.client.operation.duration`	Histogram	`s`
`gen_ai.client.inference.usage.input_tokens`	Counter	`{token}`
`gen_ai.client.inference.usage.output_tokens`	Counter	`{token}`
`gen_ai.client.inference.operation.input_tokens`	Histogram	`{token}`
`gen_ai.client.inference.operation.output_tokens`	Histogram	`{token}`

Duration covers the complete Driver run, including any retries. A failed run records duration with error.type set to the exception's fully qualified class name. Token metrics use only the final successful response's reported usage, including the final cumulative usage of a stream. Usage from failed retry attempts is unavailable and is not counted. Missing or negative token counts are omitted; explicitly reported zero counts are preserved.

All metrics include gen_ai.operation.name, gen_ai.provider.name, and gen_ai.request.model. Built-in providers use the corresponding OpenTelemetry provider name when one is defined, and custom subclasses inherit their built-in provider's classification. Other custom Drivers use their class name as the provider and chat as the operation. Google uses generate_content; Hugging Face and SageMaker use text_completion. Token counters additionally include gen_ai.token.modality="unknown", because Griptape's usage totals do not distinguish token modalities. Per-operation histograms do not include modality. Prompt and response content are not added to metric attributes.

The supplied MeterProvider is flushed when the Observability context exits, but is not shut down, allowing it to be reused. The caller owns its resource attributes, readers, exporters, and shutdown. The global meter provider's lifecycle remains managed by the application. Configure OpenTelemetry Views on the provider to customize histogram buckets; the GenAI conventions recommend duration buckets of [0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12, 10.24, 20.48, 40.96, 81.92].

These metrics do not measure streaming time to first chunk or per-chunk latency. The token metric conventions are currently in development.