Dapr AI Client

The Dapr AI client package allows you to interact with the AI capabilities provided by the Dapr sidecar.

Lifetime management

A DaprConversationClient is a version of the Dapr client that is dedicated to interacting with the Dapr Conversation API. It can be registered alongside a DaprClient and other Dapr clients without issue.

It maintains access to networking resources in the form of TCP sockets used to communicate with the Dapr sidecar.

For best performance, create a single long-lived instance of DaprConversationClient and provide access to that shared instance throughout your application. DaprConversationClient instances are thread-safe and intended to be shared.

This can be aided by utilizing the dependency injection functionality. The registration method supports registration as a singleton, a scoped instance or as transient (meaning it’s recreated every time it’s injected), but also enables registration to utilize values from an IConfiguration or other injected services in a way that’s impractical when creating the client from scratch in each of your classes.

Avoid creating a DaprConversationClient for each operation.

Configuring DaprConversationClient via DaprConversationClientBuilder

A DaprConversationClient can be configured by invoking methods on the DaprConversationClientBuilder class before calling .Build() to create the client itself. The settings for each DaprConversationClient are separate and cannot be changed after calling .Build().

var daprConversationClient = new DaprConversationClientBuilder()
    .UseDaprApiToken("abc123") // Specify the API token used to authenticate to other Dapr sidecars
    .Build();

The DaprConversationClientBuilder contains settings for:

• The HTTP endpoint of the Dapr sidecar
• The gRPC endpoint of the Dapr sidecar
• The JsonSerializerOptions object used to configure JSON serialization
• The GrpcChannelOptions object used to configure gRPC
• The API token used to authenticate requests to the sidecar
• The factory method used to create the HttpClient instance used by the SDK
• The timeout used for the HttpClient instance when making requests to the sidecar

The SDK will read the following environment variables to configure the default values:

• DAPR_HTTP_ENDPOINT: used to find the HTTP endpoint of the Dapr sidecar, example: https://dapr-api.mycompany.com
• DAPR_GRPC_ENDPOINT: used to find the gRPC endpoint of the Dapr sidecar, example: https://dapr-grpc-api.mycompany.com
• DAPR_HTTP_PORT: if DAPR_HTTP_ENDPOINT is not set, this is used to find the HTTP local endpoint of the Dapr sidecar
• DAPR_GRPC_PORT: if DAPR_GRPC_ENDPOINT is not set, this is used to find the gRPC local endpoint of the Dapr sidecar
• DAPR_API_TOKEN: used to set the API token

Configuring gRPC channel options

Dapr’s use of CancellationToken for cancellation relies on the configuration of the gRPC channel options. If you need to configure these options yourself, make sure to enable the ThrowOperationCanceledOnCancellation setting.

var daprConversationClient = new DaprConversationClientBuilder()
    .UseGrpcChannelOptions(new GrpcChannelOptions { ThrowOperationCanceledOnCancellation = true })
    .Build();

Using cancellation with DaprConversationClient

The APIs on DaprConversationClient perform asynchronous operations and accept an optional CancellationToken parameter. This follows a standard .NET practice for cancellable operations. Note that when cancellation occurs, there is no guarantee that the remote endpoint stops processing the request, only that the client has stopped waiting for completion.

When an operation is canceled, it can throw an OperationCanceledException.

Configuring DaprConversationClient via dependency injection

Using the built-in extension methods for registering the DaprConversationClient in a dependency injection container can provide the benefit of registering the long-lived service a single time, centralize complex configuration and improve performance by ensuring similarly long-lived resources are re-purposed when possible (e.g. HttpClient instances).

There are three overloads available to give the developer the greatest flexibility in configuring the client for their scenario. Each of these will register the IHttpClientFactory on your behalf if not already registered, and configure the DaprConversationClientBuilder to use it when creating the HttpClient instance in order to re-use the same instance as much as possible and avoid socket exhaustion and other issues.

In the first approach, there’s no configuration done by the developer and the DaprConversationClient is configured with the default settings.

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddDaprConversationClient(); //Registers the `DaprConversationClient` to be injected as needed
var app = builder.Build();

Sometimes the developer will need to configure the created client using the various configuration options detailed above. This is done through an overload that passes in the DaprConversationClientBuilder and exposes methods for configuring the necessary options.

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddDaprConversationClient((_, daprConversationClientBuilder) => {
   //Set the API token
   daprConversationClientBuilder.UseDaprApiToken("abc123");
   //Specify a non-standard HTTP endpoint
   daprConversationClientBuilder.UseHttpEndpoint("http://dapr.my-company.com");
});

var app = builder.Build();

Finally, it’s possible that the developer may need to retrieve information from another service in order to populate these configuration values. That value may be provided from a DaprClient instance, a vendor-specific SDK or some local service, but as long as it’s also registered in DI, it can be injected into this configuration operation via the last overload:

var builder = WebApplication.CreateBuilder(args);

//Register a fictional service that retrieves secrets from somewhere
builder.Services.AddSingleton<SecretService>();

builder.Services.AddDaprConversationClient((serviceProvider, daprConversationClientBuilder) => {
    //Retrieve an instance of the `SecretService` from the service provider
    var secretService = serviceProvider.GetRequiredService<SecretService>();
    var daprApiToken = secretService.GetSecret("DaprApiToken").Value;

    //Configure the `DaprConversationClientBuilder`
    daprConversationClientBuilder.UseDaprApiToken(daprApiToken);
});

var app = builder.Build();

Sending a conversation request

Use ConversationInput and ConversationOptions to send prompts to your conversation component:

using Dapr.AI.Conversation;
using Dapr.AI.Conversation.ConversationRoles;

var inputs = new[]
{
    new ConversationInput(new IConversationMessage[]
    {
        new SystemMessage { Content = [new MessageContent("You are a helpful assistant.")] },
        new UserMessage { Content = [new MessageContent("Explain Dapr service invocation briefly.")] }
    })
};

var options = new ConversationOptions("my-conversation-component")
{
    Temperature = 0.2
};

var response = await daprConversationClient.ConverseAsync(inputs, options);

foreach (var output in response.Outputs)
{
    foreach (var choice in output.Choices)
    {
        Console.WriteLine($"{choice.Index}: {choice.Message.Content} ({choice.FinishReason})");
    }
}

Each ConversationInput contains a list of role-specific messages. The SDK supports SystemMessage, DeveloperMessage, UserMessage, AssistantMessage, and ToolMessage. Content consists of text MessageContent entries; this API does not expose image/audio message parts.

Each response output contains Choices, optional Model, and optional Usage. Each choice has an Index, nullable FinishReason, and a Message with text and a list of tool calls. Do not assume the response always contains a single text choice.

Request options and conversation context

The client sends these options to the runtime; provider support varies. A context ID is not a client-side message history store. Keep/send any history your provider requires. ConverseAsync returns a complete response; this client has no token-streaming API in this release.

Response format (JSON schema)

You can provide a JSON schema to coerce responses into a specific format using ConversationOptions.ResponseFormat. This feature requires Dapr runtime v1.17.0 or later.

using Google.Protobuf.WellKnownTypes;

var responseFormat = new Struct
{
    Fields =
    {
        ["type"] = new Value { StringValue = "object" },
        ["properties"] = new Value
        {
            StructValue = new Struct
            {
                Fields =
                {
                    ["answer"] = new Value
                    {
                        StructValue = new Struct
                        {
                            Fields =
                            {
                                ["type"] = new Value { StringValue = "string" }
                            }
                        }
                    }
                }
            }
        },
        ["required"] = new Value
        {
            ListValue = new ListValue { Values = { new Value { StringValue = "answer" } } }
        }
    }
};

var options = new ConversationOptions("my-conversation-component")
{
    ResponseFormat = responseFormat
};

Prompt cache retention

If your conversation component supports prompt caching, you can request a cache retention window using ConversationOptions.PromptCacheRetention. This feature requires Dapr runtime v1.17.0 or later.

var options = new ConversationOptions("my-conversation-component")
{
    PromptCacheRetention = TimeSpan.FromMinutes(30)
};

Token usage statistics

The response includes optional token usage statistics when supported by the component and runtime. Usage data is available on ConversationResponseResult.Usage and includes both totals and detailed breakdowns:

var response = await daprConversationClient.ConverseAsync(inputs, options);
var result = response.Outputs[0];

if (result.Usage is not null)
{
    var totalTokens = result.Usage.TotalTokens;
    var promptCachedTokens = result.Usage.PromptTokensDetails?.CachedTokens;
    var reasoningTokens = result.Usage.CompletionTokensDetails?.ReasoningTokens;
}