Azure Data Lake Storage Gen2

Configuration for the Azure Data Lake Storage Gen2 Binary Store component

The Azure Data Lake Storage Gen2 Binary Store stores each file in the configured filesystem. The component is alpha.

Component format

The following example authenticates with a connection string:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: data-lake-files
spec:
  type: binarystore.azure.datalake
  version: v1
  metadata:
  - name: connectionString
    value: "<AZURE_STORAGE_CONNECTION_STRING>"
  - name: fileSystemName
    value: "binary-files"

Authentication

The component supports three authentication methods:

  • Connection string: provide connectionString.
  • Account key: provide accountName and accountKey.
  • Microsoft Entra ID: provide accountName and configure credentials for the sidecar’s environment. Supported credential types include workload identity, managed identity, client secret, and certificate. See Authenticating to Azure.

Spec metadata fields

FieldRequiredDetailsExample
connectionStringY for connection-string authenticationAzure Storage connection string."DefaultEndpointsProtocol=https;AccountName=mystorageaccount;AccountKey=<key>;EndpointSuffix=core.windows.net"
accountNameY for account-key and Entra ID authenticationStorage account name."mystorageaccount"
accountKeyY for account-key authenticationBase64-encoded primary or secondary storage account key."<base64-encoded-key>"
fileSystemNameYName of the Data Lake Storage Gen2 filesystem to use."binary-files"
prefixNPrefix prepended to each file name. If configured, it must not start with /."uploads"
partSizeNUpload chunk size, specified as a byte count or byte-size string. Defaults to 16Mi; must be between 1 byte and 4000 MiB. Larger chunks reduce request overhead but use more memory per in-flight upload. The maximum file size remains 5 TB."64Mi"
retryCountNMaximum number of HTTP retries for transient failures. Defaults to 3."5"
endpointNCustom endpoint URL."https://mystorageaccount.dfs.core.windows.net"
disableEntityManagementNDefaults to false. When true, the component does not try to create the filesystem during initialization; pre-create it and grant the identity read/write access."true"

Size values can be provided as a string containing the number of bytes (for example, "16777216") or as a size with a unit such as "16Mi".