This issue tracks a discussion on how we should proceed with supporting defining and managing Cosmos DB containers for an overall positive DX experience.
Background
In #4780, we added Cosmos DB support with managed identity support. In the azd add, experience, the user is able to create a cosmos DB account + database. However, the handling of containers (tables) within the Cosmos DB database is left undefined, which is important when managed identity is enabled.
The gist of the issue is: Creating containers within a Cosmos DB database is currently considered a control plane operation. Therefore, for someone who is using the existing Cosmos SDKs to use functionality like database.CreateContainerIfNotExists, or using repository-based DAO frameworks with possible schema annotation and auto-declaration fails immediately with an error like:
Operation 'POST' on resource 'colls' is not allowed through Azure Cosmos DB endpoint. Please switch on such operations for your account, or perform this operation through Azure Resource Manager, Azure Portal, Azure CLI or Azure Powershell
As a workaround, the user is able to create container using "control-plane" SDKs, they are available as samples here: https://github.com/AzureCosmosDB/management-sdk-samples . However, this requires both additional security access, and minimally needing to reference subscription ID, resource group, and cosmos database account name.
Solutions
Based on the current premise, the ecosystem it pushing us towards consideration of Solution 1 below, having schema definition defined in IaC.
Solution 2 of restoring access keys is briefly mentioned as an option to restore the existing dev-friendly, but non-infra-friendly behavior.
Solution 1: Define schema definition in IaC
If we simply did the minimum and managed it in azure.yaml, it could look like this minimally:
cappdb:
type: db.cosmos
containers:
- name: db
partitionKeys:
- /idThis may be the right thing to do for a simplified experience, and for dev-experimentation purposes. However, note that partition keys defined upfront cannot be changed currently without additional migration steps. My observation is that an ideal DX for a user onboarding journey should push the user towards some level of database planning consideration before making a schema design decision. However, with databases being managed in the same "IaC" provisioning phase -- this feels like we're merging database schema decisions upfront into the initial infrastructure standup option. A phased deployment model seems a little more tenable in this world.
It is also important to note that managing of indexes and partition keys seems a tall task within azure.yaml, and best suited as an infrastructure configuration detail. If we view it through this lens, this perhaps also pushes us towards consideration of managing infra configuration (not just app-level configuration) before full Bicep generation.*
*As a small footnote, it is important to understand that once Bicep is generated, azd will be find it extremely difficult if not impossible to resynthesize infrastructure correctly).
Solution 2: Reconsider access keys in CosmosDB
This would offer the best flexibility for the app developer, and from my perspective, the existing data plane SDKs would not expose "sharp edges" that are slightly harder to understand based on an existing DBMS conceptual model. However, it does end up exposing key management as an important security detail.
It is unclear how existing applications that leverage DAO-based configuration patterns are working today with CosmosDB in managed identity mode. My understanding is that they do not work and I am unsure about the current path forward for users are.
This issue tracks a discussion on how we should proceed with supporting defining and managing Cosmos DB containers for an overall positive DX experience.
Background
In #4780, we added Cosmos DB support with managed identity support. In the
azd add, experience, the user is able to create a cosmos DB account + database. However, the handling of containers (tables) within the Cosmos DB database is left undefined, which is important when managed identity is enabled.The gist of the issue is: Creating containers within a Cosmos DB database is currently considered a control plane operation. Therefore, for someone who is using the existing Cosmos SDKs to use functionality like
database.CreateContainerIfNotExists, or using repository-based DAO frameworks with possible schema annotation and auto-declaration fails immediately with an error like:As a workaround, the user is able to create container using "control-plane" SDKs, they are available as samples here: https://github.com/AzureCosmosDB/management-sdk-samples . However, this requires both additional security access, and minimally needing to reference subscription ID, resource group, and cosmos database account name.
Solutions
Based on the current premise, the ecosystem it pushing us towards consideration of Solution 1 below, having schema definition defined in IaC.
Solution 2 of restoring access keys is briefly mentioned as an option to restore the existing dev-friendly, but non-infra-friendly behavior.
Solution 1: Define schema definition in IaC
If we simply did the minimum and managed it in
azure.yaml, it could look like this minimally:This may be the right thing to do for a simplified experience, and for dev-experimentation purposes. However, note that partition keys defined upfront cannot be changed currently without additional migration steps. My observation is that an ideal DX for a user onboarding journey should push the user towards some level of database planning consideration before making a schema design decision. However, with databases being managed in the same "IaC" provisioning phase -- this feels like we're merging database schema decisions upfront into the initial infrastructure standup option. A phased deployment model seems a little more tenable in this world.
It is also important to note that managing of indexes and partition keys seems a tall task within
azure.yaml, and best suited as an infrastructure configuration detail. If we view it through this lens, this perhaps also pushes us towards consideration of managing infra configuration (not just app-level configuration) before full Bicep generation.**As a small footnote, it is important to understand that once Bicep is generated,
azdwill be find it extremely difficult if not impossible to resynthesize infrastructure correctly).Solution 2: Reconsider access keys in CosmosDB
This would offer the best flexibility for the app developer, and from my perspective, the existing data plane SDKs would not expose "sharp edges" that are slightly harder to understand based on an existing DBMS conceptual model. However, it does end up exposing key management as an important security detail.
It is unclear how existing applications that leverage DAO-based configuration patterns are working today with CosmosDB in managed identity mode. My understanding is that they do not work and I am unsure about the current path forward for users are.