Hi Cloud Posse team,
At the beginning of this year, we started a successful migration away from a monolithic Terraform setup toward an Atmos-based component architecture. At this point, we already have more than 100 internal components used across our engineering teams.
One thing we approached differently from the patterns I’ve seen in Atmos/Cloud Posse is dependency resolution between components.
I’ve noticed that Atmos/Cloud Posse patterns make fairly extensive use of Terraform remote state to pass information between components. In our setup, we managed to avoid remote-state dependencies entirely by using provider data sources together with a tag-based resource discovery convention.
For example, an EKS cluster is tagged with a logical discovery identity like this:
eks_discovery_tags = {
atmos_discovery_namespace = var.namespace
atmos_discovery_project = var.project
atmos_discovery_environment = var.environment
atmos_discovery_atmos_stack = local.eks_discovery_stack
atmos_discovery_resource_type = "eks"
atmos_discovery_resource_key = "default"
}
Consumers then discover the actual resource through the cloud provider API using these tags, rather than reading outputs from another component’s Terraform state.
In practice, this gives us a resource-discovery contract based on logical identity, while the cloud control plane remains the source of truth for resolving the actual resource. Terraform state is used for ownership and lifecycle management, but not as the communication mechanism between components.
We also use the concept of shared resources. A stack can act as a shared stack and provide infrastructure resources that are consumed by multiple other stacks. Those consumers do not need a direct dependency on the shared stack’s Terraform state. Instead, they discover the resources through the same logical discovery convention.
For example, a shared stack may provide an EKS cluster, VPC, subnets, Route53 zones, or other common infrastructure. Multiple application stacks can then reference those resources by their logical identity without needing to know which Terraform component created them or where its state is stored.
Conceptually, this gives us a relationship like:
shared infrastructure stack
|
+-- EKS
+-- VPC
+-- subnets
+-- Route53
|
v
cloud provider API + discovery tags
|
+--> application stack A
+--> application stack B
+--> application stack C
This has been particularly useful for keeping shared infrastructure reusable without introducing explicit state-level coupling between the shared stack and every consumer.
Another side effect we found particularly useful is that our Atmos component definitions remain relatively clean. They describe the actual configuration of the component without also having to encode explicit remote-state relationships to its dependencies.
For example, our component definitions look roughly like this:
components:
terraform:
argocd:
source:
uri: git::<ssh://git@example.org/platform/platform-components.git//components/argocd>
version: argocd-v1.7.1
vars:
health_check: true
route53_zone_name: <http://dev.example.com|dev.example.com>
git_repos:
- url: '{{ .settings.project_repository }}'
ssh_key_s3_bucket: tf-state-dev
ssh_key_s3_key: /secrets/...
provision:
workdir:
enabled: true
Dependencies such as the EKS cluster, VPC, subnets, or other infrastructure can be resolved by the component itself through the discovery convention, instead of requiring the stack definition to explicitly wire outputs from other Terraform states.
I’m curious whether Cloud Posse considered this kind of resource-discovery approach when designing the Atmos component model.
What were the main reasons for preferring remote-state/output-based dependencies between components instead?