Photo by Bryan Goff on Unsplash.
TL;DR: I've tried everything I can to get Dapr to talk to the CosmosDB Emulator container, which ships with a self-signed certificate and which therefore fails making an https connection, and I'm giving up for now.
If I find a solution to this, I'll write another blog post about it and link to it here. If there's no link, 🤷♂️.
So... I'm working on my source control system, Grace, and the next thing I'd like to do is to enable contributors (and myself) to work on it using a Docker Compose configuration.
Grace is built using Dapr, which allows users to write code once, and then plug in over 100 different pieces of infrastructure, including cloud PaaS services, so that the platform can be chosen at runtime. I love lots of things about this, but one of the big ones is that you can imagine writing your code locally running on containers for state store, observability, secrets, pub/sub, etc. and then run that same exact code in production on completely different, larger, scalable infrastructure.
Right now, Grace is sort-of hard-coded to expect Azure CosmosDB as the state store under Dapr. There was a reason in earlier versions of Dapr that I needed to do this, but Dapr has since added the features I need to remove that specific code and replace it with higher-level Dapr queries.
As part of the process of getting a Docker Compose setup going, I figured, hey, CosmosDB has an emulator container available. Might as well add that to Compose so it's there while I spend the time to rewrite the CosmosDB-specific parts into Dapr code.
The CosmosDB emulator container ships expecting to be connected to over https, which is a reasonable thing these days, but it ships with a self-signed certificate, which makes connecting over https a bit challenging. The certificate it has is not sourced from a root CA, and, by default, every browser and HTTP stack will refuse to connect to a site with a broken chain-of-trust, and self-signed certificates do not have a chain-of-trust. This is same problem we run into when, for instance, we spin up a new web project, configure it to run on https, and then get a warning message in the browser when we try to connect to it.
In my F# code, I can bypass this by configuring my CosmosClient with a custom HttpClientFactory that provides HttpClient instances that bypass TLS checking. Because this is something that you never want to do in production, I've configured it to only happen in debug builds.
let CosmosClient() =
if not <| isNull cosmosClient then
cosmosClient
else
let cosmosDbConnectionString = Environment.GetEnvironmentVariable(Constants.EnvironmentVariables.AzureCosmosDBConnectionString)
let cosmosClientOptions = CosmosClientOptions(
ApplicationName = Constants.GraceServerAppId,
EnableContentResponseOnWrite = false,
LimitToEndpoint = true,
Serializer = new CosmosJsonSerializer(Constants.JsonSerializerOptions))
#if DEBUG
let httpClientFactory = fun () ->
let httpMessageHandler: HttpMessageHandler = new HttpClientHandler(
ServerCertificateCustomValidationCallback = (fun _ _ _ _ -> true))
new HttpClient(httpMessageHandler)
cosmosClientOptions.HttpClientFactory <- httpClientFactory
cosmosClientOptions.ConnectionMode <- ConnectionMode.Direct
#endif
cosmosClient <- new CosmosClient(cosmosDbConnectionString, cosmosClientOptions)
cosmosClient
And in my Dapr config for a state store using Azure CosmosDB, I can specify the values to connect to the emulator.
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: actorStorage
spec:
type: state.azure.cosmosdb
version: v1
initTimeout: 1m
metadata:
- name: url
value: https://localhost:8081/
# This key is a well-known default value for the emulator.
- name: masterKey
value: C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
- name: database
value: gracevcs-development-db
- name: collection
value: grace-development
- name: partitionKey
value: "/partitionKey"
- name: actorStateStore
value: "true"
Cool so far. The problem comes when Dapr starts and tries to connect to the emulator.
{"app_id":"grace","instance":"dd6c98246b75","level":"fatal","msg":"process component actorStorage error: [INIT_COMPONENT_FAILURE]: initialization error occurred for actorStorage (state.azure.cosmosdb/v1): Get \"https://localhost:8081/dbs/gracevcs-development-db/colls/grace-development\": dial tcp 127.0.0.1:8081: connect: connection refused","scope":"dapr.runtime","time":"2022-12-03T01:08:37.581552869Z","type":"log","ver":"1.9.4"}
And that connection refused error is happening because of the self-signed cert on the emulator. Sigh.
I'm running on Windows 11, using WSL 2 and Docker Desktop in Linux mode. I've tried to do the things listed in Microsoft's documentation for the emulator container, including exporting the cert from the emulator and installing it into Linux. But, still, the connection is failing.
I don't have any way to tell Dapr to bypass TLS checking on connections, and I don't think they should have a flag in their code to even enable that. It's bad practice, I get that, and I don't want to give anyone a footgun they'll regret using.
Usually you write a blog post announcing how awesome you are for having figured something out.
In this case, I'm just admitting defeat for now. Just wanted this up here in case anyone else is searching for information on this. I might revisit this in a few weeks, but, for now, I'm going to try using the installed CosmosDB Emulator, and if that doesn't work quickly, I'll just keep using actual CosmosDB until I can replace the code that needs it and just use Dapr's functionality.