37 messages
👽️
P
PePe Amengual3 months ago
how this https://aws.amazon.com/blogs/security/simplified-developer-access-to-aws-with-aws-login compares to atmos login?
J
Jonathan Rose3 months ago
Random question...how could I leverage Atmos to show config drift during Bitbucket PRs that are posted as comments?
J
Jonathan Rose3 months ago(edited)
anyone else experiencing issues using
!terraform.state with mock outputs where it intermittently gives null value instead of using mock value?E
M
Mike3 months ago(edited)
I'm looking to understand the vendoring feature and its relationship to terraform module's source feature. When I think about the terraform-provider-context which is suppose to replace the null-label module, my understanding is that I'll need to write my own TF file to pull terraform-provider-context but then if I want to pass the context object or label around, I'll need to write separate tf files that wrap over components like aws-account?
This is opposed to if I use atmos vendor pull and list out the pre-built aws components then just need to write the config files/ stack file?
This is opposed to if I use atmos vendor pull and list out the pre-built aws components then just need to write the config files/ stack file?
M
Miguel Zablah3 months ago
Hi guys! I have a question how do you guys manage multiple orgs with the same structure/stacks do you just copy/paste the atmos to create a new repo per org? or change the default stack naming to just add the org as a prefix?
I was thinking that maybe having a switch between org can be cool but maybe this is just out of scope.
I was thinking that maybe having a switch between org can be cool but maybe this is just out of scope.
E
erik3 months ago
Terragrunt converts: if you could have started using atmos from day one, just by doing something this in your stack configs, would that have accelerated migration/adoption?
This would then get imported automatically into native atmos, and executed by atmos, not atmos calling terragrunbt. You could then systematically move things out of terragrunt, and into stack configs, or just leave it this way for legacy infra. If something doesn't work, you'd be able to continue using terragrunt because it's not changing the terragrunt configs.
import:
# import terragrunt units
- <terragrunt://my-terragrunt-dir/infra-live/vpc/terragrunt.hcl>
- <terragrunt://my-terragrunt-dir/infra-live/iam/terragrunt.hcl>
- <terragrunt://my-terragrunt-dir/infra-live/eks/terragrunt.hcl>
# or import a terragrunt stack
- <terragrunt://myterragrunt/infra-live/terragrunt.stack.hcl>
components:
terraform:
vpc:
vars: ... anything you want to start managing in atmos
iam:
vars: ... anything you want to start managing in atmos
eks:
vars: ... anything you want to start managing in atmosThis would then get imported automatically into native atmos, and executed by atmos, not atmos calling terragrunbt. You could then systematically move things out of terragrunt, and into stack configs, or just leave it this way for legacy infra. If something doesn't work, you'd be able to continue using terragrunt because it's not changing the terragrunt configs.
P
PePe Amengual3 months ago(edited)
I have a catalog I use everywhere but now I have a case where I need the catalog to be a bit different for qa and dev, is it possible to conditionally add an import or use a yaml function to match a value as if that is present then add xxxxx lines?
components:
terraform:
functionappwindows:
metadata:
type: abstract
vars:
name: pepe
# If is dev or QA add :
ip_whitelist:
- 2.2.2.2
- 1.1.1.1L
lefev3 months ago
Hey all! I'm having some trouble with vendoring components from https://github.com/cloudposse-terraform-components.
I've got the following
Which has led to the following directory structure:
This has the following issues:
• All the terraform files are in a
• The acm component has some sort of implicit dependency on
Is there any way to prevent vendoring placing the files under the
Or is this just a bug in the
I've got the following
vendor.yaml:# Vendor manifest for component management
apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
name: component-vendoring
description: Vendor configuration for infrastructure components
spec:
# Source specifications
sources:
- component: acm
source: "<http://github.com/cloudposse-terraform-components/aws-{{.Component}}.git?ref={{.Version}}|github.com/cloudposse-terraform-components/aws-{{.Component}}.git?ref={{.Version}}>"
version: "v2.0.0"
targets:
- "components/terraform/{{.Component}}"
included_paths:
- "**/*.tf"
- "**/*.tfvars"
- "README.md"
- "LICENSE"
excluded_paths:
- "examples/**"
- "test/**"
- ".github/**"
- component: account-map
source: "<http://github.com/cloudposse-terraform-components/aws-{{.Component}}.git?ref={{.Version}}|github.com/cloudposse-terraform-components/aws-{{.Component}}.git?ref={{.Version}}>"
version: "v1.536.1"
targets:
- "components/terraform/acm/{{.Component}}"
included_paths:
- "**/*.tf"
- "**/*.tfvars"
- "README.md"
- "LICENSE"
excluded_paths:
- "examples/**"
- "test/**"
- ".github/**"Which has led to the following directory structure:
components/terraform/acm/
├── account-map
│ ├── README.md
│ └── src
| └── ... Omitted ...
├── backend.tf.json
├── providers_override.tf.json
├── README.md
└── src
├── backend.tf.json
├── context.tf
├── ... Omitted ...This has the following issues:
• All the terraform files are in a
src/ subdirectory which means I need to reference the components as acm/src . Not a huge deal, just annoying, but leads to the bigger issue below.• The acm component has some sort of implicit dependency on
account-map expecting it to be at ../account-map. This may be a bug in of itself but my attempt to fix this by vendoring account-map component inside the acm component hasn't worked due to the vendoring putting the files inside src/Is there any way to prevent vendoring placing the files under the
src/ directory? If not is the expected way to use vendored components by defining them as component: acm/src?Or is this just a bug in the
acm component that needs to be fixed?E
erik3 months ago
@PePe Amengual this one's for you: ❤️
https://atmos.tools/cli
• Every section of the stack configuration is documented
• Every section of the atmos.yaml is documented
• Every section of the vendor.yaml is documented
• Every section of the component.yaml (component manifest) is documented
• Added some small missing commands that were undocumented
• All commands now link to their respective configuration documentation
• All configurations link to their respective commands
https://atmos.tools/cli
• Every section of the stack configuration is documented
• Every section of the atmos.yaml is documented
• Every section of the vendor.yaml is documented
• Every section of the component.yaml (component manifest) is documented
• Added some small missing commands that were undocumented
• All commands now link to their respective configuration documentation
• All configurations link to their respective commands
E
erik3 months ago
For those who've spent more time with Terragrunt, I'd appreciate feedback on how to make this better: https://atmos.tools/migration/terragrunt
R
RB3 months ago
quick question on atmos pro, does it comment on github PRs like atlantis or does it use github’s Checks API to post plan/apply output like spacelift ?
J
Jonathan Rose2 months ago(edited)
Does atmos have a way to provide configurations that are implied to be provided a YAML encoded string? Or does the root module need to leverage
yamlencode()?R
Ricky Fontaine2 months ago(edited)
Hi! This morning I've encountered the following error in my Terraform Plan CI on GitHub actions, in the
The particular PR only changes a component in a different stack. Is there a problem with the terraform registry? Do I need to update the version of awsutils? Or is there something else that could be causing this? Thanks!
Determine Affected Stacks step. ✗ Fetching eks_cluster_managed_security_group_id output from eks/cluster in plat-ue1-prod
# Error
failed to execute terraform output for the component eks/cluster in the stack plat-ue1-prod. Error: exit status 1
Error: Failed to install provider
Error while installing cloudposse/awsutils v0.19.1: could not query provider registry for registry.terraform.io/cloudposse/awsutils: failed to retrieve cryptographic signature for provider: the request failed after 2 attempts, please try again later: 503 Service Unavailable returned from github.com The particular PR only changes a component in a different stack. Is there a problem with the terraform registry? Do I need to update the version of awsutils? Or is there something else that could be causing this? Thanks!
R
Ricky Fontaine2 months ago
Potentially related to the above, is atmos affected at all by Terraform sunsetting its CDK? https://github.com/hashicorp/terraform-cdk?tab=readme-ov-file#sunset-notice
C
cromega2 months ago(edited)
Hi, I may have updated atmos yesterday to 201, not sure, in any case, I get a new error while loading atmos for the following part in my stack, this is right under vars in a component:
and I get
I have identical subnet id queries elsewhere in the stack, no idea why this one specifically triggers something. if I replace the terraform state stuff with just "subnet-1234", it works. downgrading to 194 solves it
overseer:
db_subnets: !terraform.state services-network ".subnets[] | select(.group == ""overseer"") | .id"and I get
Error: failed to set value at overseer.db_subnets: cannot navigate path: field is
not a map: path=[overseer db_subnets] field=overseer I have identical subnet id queries elsewhere in the stack, no idea why this one specifically triggers something. if I replace the terraform state stuff with just "subnet-1234", it works. downgrading to 194 solves it
N
Nick Dunn2 months ago(edited)
Hi, everyone. I plan to submit a GH issue, but I wanted to make sure I'm not crazy. I believe I may have stumbled across a strange bug with a misleading error message.
My problem can be reproduced relatively easily. I have a stack:
And then I create a component called "testme" with a single
You'll notice the typo in the
So, if I run
If I remove the syntax mistake, the problem completely disappears and everything runs as expected. The only changes I've made to Atmos recently is I added an 'auth' block to our configuration so we could take advantage of those features. Is this a legitimate bug that I should create a GH Issue for or have I overlooked/misunderstood something? I've definitely had Terraform syntax errors before that did not cause this behavior. Are Terraform locals treated differently in Atmos under the hood?
My problem can be reproduced relatively easily. I have a stack:
components:
terraform:
testme: {}And then I create a component called "testme" with a single
main.tf file.locals {
hiWorld = "hello world"
]
}
output "hw" {
value = local.hiWorld
}You'll notice the typo in the
main.tf. It took me quite a while to eventually boil the problem down to this. To be fair, I should have noticed the syntax error sooner, but I figured if the problem was bad Terraform, I'd get an error message stating that. So, I didn't really bother looking at the component's Terraform at first.So, if I run
atmos terraform plan or even atmos terraform validate for this component, I get this: Error
Error: invalid component
## Explanation
Could not find the component testme in the stack weird-problem.
Check that all the context variables are correctly defined in the stack manifests.
Are the component and stack names correct? Did you forget an import?If I remove the syntax mistake, the problem completely disappears and everything runs as expected. The only changes I've made to Atmos recently is I added an 'auth' block to our configuration so we could take advantage of those features. Is this a legitimate bug that I should create a GH Issue for or have I overlooked/misunderstood something? I've definitely had Terraform syntax errors before that did not cause this behavior. Are Terraform locals treated differently in Atmos under the hood?
E
erik2 months ago
did you know you can now run atmos terraform plan /some/path/to/a/component?• If multiple instances of that component exist, it will prompt you to select which one
• This makes it easier because you don't need to know the component name in the stack
https://atmos.tools/changelog/path-based-component-resolution
Z
Zack2 months ago
@Erik Osterman (Cloud Posse) do you guys have an example of a "mature" atmos repo publicly? We're just starting to run into versioning issues.. re-structing of our
vendor.yaml (renovate bit us) etc and looking at how to implement properly with abstract includesP
PePe Amengual2 months ago(edited)
did something changes with basepath detection?
export ATMOS_CLI_CONFIG_PATH="./config" export ATMOS_BASE_PATH="./"
pepe.amengual@pepe ~/g/a/pepe-iac (feature/setup-docker-config-json-for-ghcr)> atmos helmfile sync actions-runner-controller -s mgmt-wus3
Error
Error: directory for Atmos stacks does not exist
## Hints
:bulb: Stacks directory not found:
/Users/pepe.amengual/github/pepe-iac/config/stacks
pepe.amengual@pepe ~/g/a/pepe-iac (feature/setup-docker-config-json-for-ghcr) [1]> atmos --version
█████ ████████ ███ ███ ██████ ███████
██ ██ ██ ████ ████ ██ ██ ██
███████ ██ ██ ████ ██ ██ ██ ███████
██ ██ ██ ██ ██ ██ ██ ██ ██
██ ██ ██ ██ ██ ██████ ███████
:alien: Atmos 1.201.0 on darwin/arm64P
PePe Amengual2 months ago
I just wanted a bash version like tfenv so I made one : https://github.com/jamengual/atmosenv/releases/tag/v0.1.0
J
Jonathan Rose2 months ago
Question for the group, a good amount of our root modules are sourced from Terraform AWS modules. By doing so, it creates warnings about unused variables normally sourced from https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf
However, if I sideload the context file, I typically have to delete the
However, if I sideload the context file, I typically have to delete the
name and tags variables to resolve a duplicate variable declaration. In addition to that, tflint flags this standalone file due to the lack of requirements defined (e.g. minimum version of terraform). Thoughts? Anyone else deal with this? Create an exception in for tflint?D
david2 months ago
How do I get atmos to ignore a gomplate? Eg, I have the following
I wish for
db_users: ["{{external.email}}"]I wish for
"{{external.email}}" to be passed in literallyM
Mihai I2 months ago
Is this documentation section correct? https://atmos.tools/cli/configuration/stores this one suggest type for store,this one https://atmos.tools/stacks/sharing-state/stores mentions configuration should use backend? Do I understand correctly that the Secrets Manager is not supported, only Parameter store?
Z
Zack2 months ago
I'm curious, how do you guys manage renovate/keeping your vendor.yaml's up-to-date?
N
Nayeem Mohammed2 months ago
Hey guys, I have been reading about Atmos vendoring and wanted to check if there are any video demos that go over the use cases etc
If you guys can please share that would be helpful. Thanks in advance
If you guys can please share that would be helpful. Thanks in advance
Y
Yota2 months ago
Hello @Erik Osterman (Cloud Posse). I think you forgot to add the release v1.202 to the page https://atmos.tools/changelog
E
erik2 months ago
Would love feedback on this upcoming atmos feature: native CI integration in atmos without using
• Status checks API (so you can create merge requirements)
• PR commments (templated and configurable)
• Job summaries (same)
• matrix generation
•
• pluggable, so adapters can be written for Gitlab/ADO/etc
cloudposse/github-action-atmos-*;• Status checks API (so you can create merge requirements)
• PR commments (templated and configurable)
• Job summaries (same)
• matrix generation
•
GITHUB_OUTPUT - use terraform ouputs between jobs• pluggable, so adapters can be written for Gitlab/ADO/etc
M
Miguel Zablahabout 2 months ago
Hey guys, I'm having an issue in version
I use asume roles with SSO logging and this works perfectly on versions before that one but since this version the role is not resolving correctly even when the
any suggestion or any changes on how atmos resolved interpolation that it might affect this?
1.202.0 and up where now the role authentication is now being resolved correctly.I use asume roles with SSO logging and this works perfectly on versions before that one but since this version the role is not resolving correctly even when the
backend.tf.json and providers.tf.json file looks good. the only thing I see that might be the issue is that I use interpolation on the org config to generate this roles and on the terminal the role is not resolving but as mention before the files are looking good.any suggestion or any changes on how atmos resolved interpolation that it might affect this?
E
erikabout 2 months ago(edited)
One of the most requested features is to managed secrets in atmos. We're starting to think about it and how we would support it. We would like to support full CRUD operations on secrets to make it as easy as using Vercel/Heroku, but using cloud backends like ASM/SSM/SOPS/etc but fully extensible (with other backends supported through a registry pattern). It will tie in to the new atmos auth identity system, so you can have different credentials to access different secrets stores, and would support GitHub OIDC out of the box (since that's already supported by atmos auth). On top of that, you'd be able to initialize secrets, with an interactive configuration flow.
The write up is here. Please feel free to leave comments.
https://github.com/cloudposse/atmos/blob/osterman/secrets-management/docs/prd/secrets-management.md
The write up is here. Please feel free to leave comments.
https://github.com/cloudposse/atmos/blob/osterman/secrets-management/docs/prd/secrets-management.md
P
PePe Amengualabout 2 months ago
is it possible to use ENV vars to feed the spec in atmos auth?
auth:
providers:
azure-dev-ci:
kind: azure/oidc
spec:
tenant_id: ${secret.tenant_id}
client_id: ${secret.client_id} # Dev SP
subscription_id: ${subscription_id}E
erikabout 2 months ago
One of my favorite features that we released this year in atmos are “profiles”. It comes with the sober realization that there is never just one context for configuration. It changes whether you are running in CI, if you are a developer working in a few environments, or a power user leveraging all that atmos has to offer. Traditionally, this would be solved by environment variables. But then how do you consistently mange those variables in each context? When new variables are introduced how are those communicated? By taking the approach of using atmos profiles you can skip convoluted work arounds. Set one environment variable and manage everything else as pure configuration.
See https://atmos.tools/changelog/atmos-profiles/
See https://atmos.tools/changelog/atmos-profiles/
E
erikabout 2 months ago
We've published the atmos roadmap! This helps tell the story of where we're headed with atmos, and what we've accomplished so far.
https://atmos.tools/roadmap
https://atmos.tools/roadmap
D
Daniel Boothabout 2 months ago
is there a way to generate the required_providers block?
D
Daniel Boothabout 2 months ago
I could only find info for providers in the atmos docs
L
Leonardo Agueciabout 2 months ago
Hi all, thank you for welcoming me and for the great work you are doing 🙂 I'm here because I need help with my Atmos configuration.
I'm using Atmos 1.198.0 to create the infrastructure in AWS via Terraform.
In my setup I'm using S3 backend, I have the terraform workspace disabled and the key set to something like <stack name>/<atmos component>.terraform.tfstate.
I'm using the function !terraform.output to retrieve the id of some components (for example in the EC2 I use it to get the Id of the security groups) and everything works correctly. However, for performance reasons, I want to use the !terraform.state function, but this function cannot read the state from the backend. From the logs, it appears that even though the workspace is disabled, when the function tries to read the state from the S3 bucket, it looks in the wrong path, specifically at _<workspace_key_prefix>/default/<stack name>/<atmos component>.terraform.tfstate._
Given that when workspaces are disabled and the default one is used, the function should not append either the workspace_key_prefix or the workspace name so that its behavior aligns with Terraform.
Is there a way to force this behavior, or is something being misconfigured?
I also tried to upgrade Atmos to the latest version (1.202.1), but the behavior remains the same.
With this version, however, it seems there are some authentication-related issues. I'm using Atmos internal authentication with AWS SSO configured as the provider. When I run a command such as atmos describe component an error like panic: authContextWrapper.GetChain should not be called is thrown. From the debug logs, the error seems related to the !terraform.output function, specifically when it tries to create the AuthManager for one of the nested components. The !terraform.state function instead does not have this problem. May someone help me on resolving these issues? Thanks a lot 🙂
I'm using Atmos 1.198.0 to create the infrastructure in AWS via Terraform.
In my setup I'm using S3 backend, I have the terraform workspace disabled and the key set to something like <stack name>/<atmos component>.terraform.tfstate.
I'm using the function !terraform.output to retrieve the id of some components (for example in the EC2 I use it to get the Id of the security groups) and everything works correctly. However, for performance reasons, I want to use the !terraform.state function, but this function cannot read the state from the backend. From the logs, it appears that even though the workspace is disabled, when the function tries to read the state from the S3 bucket, it looks in the wrong path, specifically at _<workspace_key_prefix>/default/<stack name>/<atmos component>.terraform.tfstate._
Given that when workspaces are disabled and the default one is used, the function should not append either the workspace_key_prefix or the workspace name so that its behavior aligns with Terraform.
Is there a way to force this behavior, or is something being misconfigured?
I also tried to upgrade Atmos to the latest version (1.202.1), but the behavior remains the same.
With this version, however, it seems there are some authentication-related issues. I'm using Atmos internal authentication with AWS SSO configured as the provider. When I run a command such as atmos describe component an error like panic: authContextWrapper.GetChain should not be called is thrown. From the debug logs, the error seems related to the !terraform.output function, specifically when it tries to create the AuthManager for one of the nested components. The !terraform.state function instead does not have this problem. May someone help me on resolving these issues? Thanks a lot 🙂
I
Igor Mabout 2 months ago
I created a few skills and hooks with Claude Code w/ Atmos, including an
Just had a user that has never done infra-as-code go through the whole flow, and create a feature flag using atmos all on their own.
Just thought I'd share that. Pretty epic stuff.
/onboard one.Just had a user that has never done infra-as-code go through the whole flow, and create a feature flag using atmos all on their own.
Just thought I'd share that. Pretty epic stuff.