Skip to main content

Datadog LLM Observability

danger

LLM Observability is in public beta, and its API is subject to change.

With Datadog LLM Observability, you can monitor, troubleshoot, and evaluate your LLM-powered applications, such as chatbots. You can investigate the root cause of issues, monitor operational performance, and evaluate the quality, privacy, and safety of your LLM applications.

This is an experimental community implementation, and it is not officially supported by Datadog. It is based on the Datadog LLM Observability API.

Setup​

npm install @langchain/community

Usage​

import { OpenAI } from "@langchain/openai";
import { DatadogLLMObsTracer } from "@langchain/community/experimental/callbacks/handlers/datadog";

/**
* This example demonstrates how to use the DatadogLLMObsTracer with the OpenAI model.
* It will produce a "llm" span with the input and output of the model inside the meta field.
*
* To run this example, you need to have a valid Datadog API key and OpenAI API key.
*/
export const run = async () => {
const model = new OpenAI({
model: "gpt-4",
temperature: 0.7,
maxTokens: 1000,
maxRetries: 5,
});

const res = await model.invoke(
"Question: What would be a good company name a company that makes colorful socks?\nAnswer:",
{
callbacks: [
new DatadogLLMObsTracer({
mlApp: "my-ml-app",
}),
],
}
);

console.log({ res });
};

API Reference:


Was this page helpful?


You can also leave detailed feedback on GitHub.