43 messages
👽️
Bart Palmowski8 months ago
Hi, I'm trying to understand this: https://github.com/cloudposse/atmos/blob/main/examples/quick-start-advanced/stacks/catalog/vpc/dev.yaml
Bart Palmowski8 months ago
Is there an example on how to use
atmos terraform import?Bart Palmowski8 months ago
Hi, I see this document specifying (more or less) the github integration: https://docs.cloudposse.com/learn/maintenance/upgrades/how-to-migrate-gitops-config-into-atmos-yaml/ the problem is that I need to run terraform that authenticates with google cloud instead of aws
Ian Cornett8 months ago(edited)
Is it common to find that none of the documented design patterns fully meet your needs, but overlaying several in a swiss-cheese approach does? For example, a multi-org, multi-cloud, multi-component architecture might use Organization + Component Catalog + Multi-Component + Abstract?
PePe Amengual8 months ago
if
!terraform.output config .mydns_domain_map.(Here I need my domain with dots) is possible to do that?Kyle Decot8 months ago
Hi there 👋. How would one go about debugging the following error:
https://atmos.tools/troubleshoot/debugging/ just says "TODO" 😢
Thanks in advance for any help!
atmos terraform apply github -s organization
Error
parse error on line 1, column 22: bare " in non-quoted-fieldhttps://atmos.tools/troubleshoot/debugging/ just says "TODO" 😢
Thanks in advance for any help!
kofi8 months ago
Hi! Is anyone getting the command
atmos describe stacks -s <stack> --format json --no-color --pager false --process-templates --process-functions becoming noticeably slow since v1.181.0?Jonathan Rose8 months ago
Is cloudposse-terraform-components/aws-tgw-routes: Manage the routes in a Transit Gateway Route Table a working solution? I am noting no releases,
Miguel Zablah8 months ago
Hey there! quick question is
vendor.yaml able to download like release zip files?Bruce8 months ago
quick question. if i wanted the output of a tf module but i want to concat it with another string, how can i do that?
i want to add
i want to add
redis:// to the the output of the redis module. redis-ssm-parameter-store:
metadata:
component: aws-ssm-parameter-store
vars:
parameter_write:
- name: "/redis/endpoint"
type: "SecureString"
value: "<redis://%s>" (!terraformOutput "elasticache-redis" "redis_clusters.redis.cluster_endpoint")Jonathan Rose7 months ago
hello, can someone confirm this is valid for enabling state lock using s3?
specifically, is
backend_type: s3
backend:
s3:
acl: "bucket-owner-full-control"
bucket: tfstate-backend
encrypt: true
key: "terraform.tfstate"
region: "us-east-1"
workspace_key_prefix: "ami-linux"
use_lockfile: truespecifically, is
use_lockfile supported? i am using Terraform 1.11.4 and documentation says it's good but unsure how to validate since its first time trying it and logging doesn't mention anything about state lockingJonathan Rose7 months ago
I'm trying to use atmos.Component | atmos to replace references to
The resulting plan takes the information provided literally. For example:
any suggestions?
!terraform.output since of performance issues. However, when I try to use it along with mock outputs, I doesn't work. For example, if I define:components:
terraform:
ami-linux:
vars:
defaults:
subnet_id: '{{ (atmos.Component "vpc" .stack).outputs.private_subnets[0] // MOCK__subnet_id }}'
security_group_ids:
- '{{ (atmos.Component "security-group" .stack).outputs.wrapper.image_builder.security_group_id // MOCK__security_group_id }}'The resulting plan takes the information provided literally. For example:
+ security_group_ids = [
+ "{{ (atmos.Component \"security-group\" .stack).outputs.wrapper.image_builder.security_group_id // MOCK__security_group_id }}",
]
+ sns_topic_arn = null
+ subnet_id = "{{ (atmos.Component \"vpc\" .stack).outputs.private_subnets[0] // MOCK__subnet_id }}"any suggestions?
Michael Pursifull7 months ago
Please consider upvoting for atmos here: https://feedback.spacelift.io/p/incorporating-atmos-into-stack-creation
Jonathan Rose7 months ago
The IAC Service Catalog that I am compiling is using terragrunt-style wrappers (e.g. terraform-aws-ecs/wrappers at v6.0.5 · terraform-aws-modules/terraform-aws-ecs). If I wanted to create a component using https://github.com/cloudposse/terraform-aws-vpn-connection/tree/v1.8.1, what would be the recommended solution?
cromega7 months ago
Hello atmos people, I'm evaluating atmos and I didn't get very far. I've set up a very simple stack with a very simple component but when I run atmos and I select the stack or the component, I get an error, the backend type in the generated backend.tf.json file is empty. It looks like atmos is not picking up the _defaults.yaml file I have under stacks. if I put the contents of that file in mystack.yaml, I seem to get the right thing.
E
erik7 months ago
Here's how you can use the Warp AI terminal (like cursor but for the terminal) with the Context7 MCP which knows atmos.
https://www.warp.dev/
https://github.com/upstash/context7
https://context7.com/cloudposse/atmos
1. Navigate
2. Add a new MCP server by clicking the
3. Paste the configuration given below:
https://www.warp.dev/
https://github.com/upstash/context7
https://context7.com/cloudposse/atmos
1. Navigate
Settings > AI > Manage MCP servers.2. Add a new MCP server by clicking the
+ Add button.3. Paste the configuration given below:
{
"Context7": {
"command": "npx",
"args": [
"-y",
"@upstash/context7-mcp"
],
"env": {},
"working_directory": null,
"start_on_launch": true
}
}cromega7 months ago(edited)
reading the docs I was wondering about stsack. I sort of understand that components should be "not too big and not too small" but I'm unsure about stacks. we don't have clients/tenants, etc. Would it be reasonable to say that a stack for us is everything that is deployed under an AWS account in a specific region so a stack would be an
account-region sort of thing, even if it means we only have 2-3 stacks altogetherJonathan Rose7 months ago(edited)
is there a way for atmos to ensure terraform-github-repository/context.tf at main · cloudposse/terraform-github-repository is included for every component i create or maintain? (e.g. vendor manifest)
MP7 months ago
Hi all, is there a trick to using
tofu state rm with atmos? I keep running into "Could not find the component" and apparent syntax errors and haven't been able to find any docs on usage with atmos. Where does the atmos component name go in the command?cromega7 months ago
The docs say that dependency management depends on integrations so just to confirm: If I have a dependency defined between 2 components in a stack, is there no way to let atmos figure it out and deploy component1 first and then component2 automatically? If I apply component2 I don't even get a warning saying "yo, component1 is not even deployed" even if it needs values from the output?
remym7 months ago
Hello,
Would it be useful to have support for default values for flags in custom CLI commands?
Let me explain — I know this feature already exists for arguments (such as positional ones), but it’s not possible, for example, to define only
The advantage of using flags is that they are prefixed (e.g.,
Would it be useful to have support for default values for flags in custom CLI commands?
Let me explain — I know this feature already exists for arguments (such as positional ones), but it’s not possible, for example, to define only
arg1 and arg3 without arg3 being interpreted as arg2.The advantage of using flags is that they are prefixed (e.g.,
--flagname), which allows them to be specified independently and avoids ambiguity.Martin7 months ago(edited)
Hey there! I'm playing around with atmos atm (awesome tool so far!) and was wondering if I'm doing things "the right way".
I have a pretty simple setup atm - 3 stacks (one
1. I was wondering though - when one catalog module depends on some output of another catalog module, should I put these directly into these modules OR should I define these in my stacks?
2. When does atmos change
Let me know if you need some code - I can create a repo if required. Thanks!
I have a pretty simple setup atm - 3 stacks (one
gbl-root and two for prod and dev)1. I was wondering though - when one catalog module depends on some output of another catalog module, should I put these directly into these modules OR should I define these in my stacks?
2. When does atmos change
workspace? because I see Switched to workspace "gbl-root-ses" but I only have gbl-root defined really in my code. I'm just wondering why it did that because everything else is in gbl-rootLet me know if you need some code - I can create a repo if required. Thanks!
Alcp7 months ago
Atmos inheritance
Does atmos prevent inheriting from real components? I mean does the base component has to be abstract to inherit?
Does atmos prevent inheriting from real components? I mean does the base component has to be abstract to inherit?
cromega7 months ago
enhancement request: could fetching states for !terraform.output be parallelised?
Jonathan Rose7 months ago
Is
!terraform.state supposed to work with YAML default values? I am getting errors. For example:# Error
failed to read Terraform state for component vpc in stack cfsb-it-platform-dev in YAML function: !terraform.state vpc "vpc_id // ""MOCK__vpc_id""" failed to get object from S3: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded cromega7 months ago
how important it is to get the stack hierarchy right? If I rename/restructure things then I lose the terraform states, will I have to manually move things around manually?
cromega7 months ago
a small bug report: filtering the stack list until there is only 1 item does not update the component list until I hit up/down
Jonathan Rose7 months ago
If I wanted to include the following as key/value pairs to define tags
What is the notation I would use?
vars:
namespace: cfsb
tenant: ops
environment: ue1
stage: devWhat is the notation I would use?
providers:
aws:
assume_role:
role_arn: "arn:aws:iam::242985636183:role/ci-terraform"
default_tags:
tags:
Environment: {.vars.stage}
Tenant: {.vars.tenant}
Namespace: {.vars.namespace}
Stage: {.vars.stage}
region: "us-east-1"
version: "5.100.0"cromega7 months ago(edited)
I got into a funny situation by moving a component and migrating the terraform state, I will try to recollect what I did:
in a stack called
in a stack called
stack, a component called component, I had this in my stack configuration: components: {terraform: {component: {}}. I moved the component into a subfolder so I added this to my component definition in the stack: metadata: {component: subfolder/component} . This made terraform prompt for a state migration, which I did by running atmos terraform init -s stack component -- -migrate-state. The command succeeded but after running plan, I saw that the state was all gone. Turns out it was migrated into the wrong folder. Looks like atmos terraform init migrated the state into subfolder-component/stack/terraform.tfstate but it was supposed to be under subfolder-component/stack-component/terraform.tfstateMiguel Zablah7 months ago
Hi team I have found a strange issue when running a Workflow on Atmos version 1.83.1 where it will run the CMDs successfully but it will still say it fails and say the cmd do not exist, this is the "error":
this is not a blocker per say since it dose the workflow correctly but it's a bug, if there is not ticket for this I can create one
Incorrect Usage
The command atmos workflow is not valid usage
## Usage Examples:
– Use interactive UI
$ atmos workflow
– Execute a workflow
$ atmos workflow <workflow-name> --file <file>
– Execute with stack override
$ atmos workflow <workflow-name> --file <file> --stack <stack>
– Resume from specific step
$ atmos workflow <workflow-name> --file <file> --from-step <step>
For more information, refer to the docs <https://atmos.tools/cli/commands/workflow/>this is not a blocker per say since it dose the workflow correctly but it's a bug, if there is not ticket for this I can create one
Miguel Zablah7 months ago(edited)
Hey do we have docs for
so !terraform.output/!terraform.state will get the secret ARN:
And I will like to append this string to get a specific secret:
so at the end the yaml will look like this:
Is there a way to do this? I have try many ways but I continue to use
!terraform.output and !terraform.state interpolations? I will like to do simple add a string at the end of the output like for secrets.so !terraform.output/!terraform.state will get the secret ARN:
arn:aws:secretsmanager:<REGION>:<AWS>:secret:<SUPER_SECRET_NAME>And I will like to append this string to get a specific secret:
:username::so at the end the yaml will look like this:
username: 'arn:aws:secretsmanager:<REGION>:<AWS>:secret:<SUPER_SECRET_NAME>:username::'Is there a way to do this? I have try many ways but I continue to use
atmos.Component bc it works better for meKev Silver7 months ago(edited)
Hi there! I'm trying to execute an Atmos workflow in Github Actions, but it hangs and never makes actual progress. The workflow itself is a short list of applies:
But never makes progress through even the first command and produces no output that I can see in Actions logs. Running the same workflow locally has no issues, and I can't get any logs to spit out to give me an idea of what's going on in the Actions runner, if anything, even with
For reference, the command I'm running is:
workflows:
my-deploy:
steps:
- command: terraform apply my-component -s my-stack/dev -auto-approve
- [...]But never makes progress through even the first command and produces no output that I can see in Actions logs. Running the same workflow locally has no issues, and I can't get any logs to spit out to give me an idea of what's going on in the Actions runner, if anything, even with
ATMOS_LOGS_LEVEL and ATMOS_LOGS_FILE set. Has anyone else run into this?For reference, the command I'm running is:
atmos workflow my-deploy -f workflow-my-deployJonathan Rose7 months ago(edited)
Question, I have two competing components overwriting the backend.json file in the same directory. How do I ensure each component (each used a common root module/abstract component), uses a unique backend.json (e.g. prefix the backend.json with the stack name)
Jonathan Rose7 months ago
Next question - how can I configure Atmos to automatically run
atmos terraform clean before running any subsequent commands?Bart Palmowski7 months ago
when I import a template I always need to provide a non-empty context or atmos will think it's a yaml file, any way I could do something like
but without
- path: catalog/gcp/vpc/macro.tmpl
context:
foo: barbut without
context?Bart Palmowski7 months ago
In examples here: https://atmos.tools/core-concepts/stacks/templates/ a
.atmos_stack is present but it does not exist for meJustin7 months ago
Hi all!
I've just completed implementing the Atmos Plan and Apply GitHub Actions as part of our deployment workflow and I'm super impressed with the functionality and quality of life increase this has provided. The feedback and output from the actions is fantastic and we have several fans at work. This has really helped improve velocity and overall confidence in our deployment pipelines so thank you!
With the deployment pipelines in place, I'm now working on splitting up a Terraform module that has significant resource count down into several smaller components. One of the core drivers for keeping everything in the same module is some of the tightness in which things are coupled. Using resource references as part of the Terraform DAG made it easy to update multiple components in one pass. However, the module is way too large now and presents the typical blast radius / deployment blocking / calculation time risks.
What I need to work on alongside this project is the ability to move these tight couplings from inside the module to the dependency configuration Atmos has built in. We're already using this structure a bit but it's only for pointing engineers to dependency patterns rather than being programmatically functional.
I understand that nested dependency plan/apply functionality is not currently capable in GitHub Actions so I'd like to reach out here for guidance. If my research is correct, it looks like I have two paths forward:
• Deploy Atlantis and update the plan/apply workflows to lean on Atlantis for planning and applying dependent modules based on get-affected-stacks changes.
• Deploy Spacelift and tie that together with Atmos. I'm more hesitant on this path as I think it will be a significantly larger lift and I'd like to have something by end of Q3 if possible.
Am I on the right track? Does anyone here have an established pattern for ensuring that if a component is marked for change, it's downstream dependent components will be marked for a subsequent plan/apply?
Thanks in advance!
I've just completed implementing the Atmos Plan and Apply GitHub Actions as part of our deployment workflow and I'm super impressed with the functionality and quality of life increase this has provided. The feedback and output from the actions is fantastic and we have several fans at work. This has really helped improve velocity and overall confidence in our deployment pipelines so thank you!
With the deployment pipelines in place, I'm now working on splitting up a Terraform module that has significant resource count down into several smaller components. One of the core drivers for keeping everything in the same module is some of the tightness in which things are coupled. Using resource references as part of the Terraform DAG made it easy to update multiple components in one pass. However, the module is way too large now and presents the typical blast radius / deployment blocking / calculation time risks.
What I need to work on alongside this project is the ability to move these tight couplings from inside the module to the dependency configuration Atmos has built in. We're already using this structure a bit but it's only for pointing engineers to dependency patterns rather than being programmatically functional.
I understand that nested dependency plan/apply functionality is not currently capable in GitHub Actions so I'd like to reach out here for guidance. If my research is correct, it looks like I have two paths forward:
• Deploy Atlantis and update the plan/apply workflows to lean on Atlantis for planning and applying dependent modules based on get-affected-stacks changes.
• Deploy Spacelift and tie that together with Atmos. I'm more hesitant on this path as I think it will be a significantly larger lift and I'd like to have something by end of Q3 if possible.
Am I on the right track? Does anyone here have an established pattern for ensuring that if a component is marked for change, it's downstream dependent components will be marked for a subsequent plan/apply?
Thanks in advance!
Erik Osterman (Cloud Posse)7 months ago
☑️ What component types do you wish atmos supported natively?
PePe Amengual7 months ago(edited)
Does describe-affected
"ignores" components with metadata.locked: true? And if it does, does it ignore the dependencies?PePe Amengual7 months ago(edited)
Atmos :1.184.0; did anything changes in new version of atmos vendor? I just updated and got
ERRO Failed to vendor github.com/pepe-org/pepe-iac.git//?ref=main: error : github.com/pepe-org/pepe-iac.git//?ref=main: failed to download package: error downloading '<https://github.com/pepe-org/pepe-iac.git?depth=1&ref=main>': /usr/bin/git exited with 128: fatal: could not read Username for '<https://github.com>': No such device or address
INFO x package="github.com/pepe-org/pepe-iac.git//?ref=main" version=(main)PePe Amengual7 months ago
the
atmos terraform plan --affected is basically a describe affected + plan? I ask because of the copy commands happening hereDEBU Set logs-level=debug logs-file=/dev/stderr
DEBU Found config ENV ATMOS_CLI_CONFIG_PATH=./atmos/config
DEBU Found ENV variable ATMOS_BASE_PATH=./atmos
DEBU Found config ENV ATMOS_CLI_CONFIG_PATH=./atmos/config
DEBU Found ENV variable ATMOS_BASE_PATH=./atmos
DEBU Found config ENV ATMOS_CLI_CONFIG_PATH=./atmos/config
DEBU Found ENV variable ATMOS_BASE_PATH=./atmos
DEBU Set using command line argument --logs-level=Debug
DEBU Atmos JSON Schema is not configured. Using the default embedded schema
DEBU Validating all YAML files in the folder and all subfolders (excluding template files) folder=atmos/stacks
DEBU Copying the local repo into temp directory dir=/tmp/3918833686
DEBU Copied the local repo into temp directory dir=/tmp/3918833686
DEBU Checking out Git ref=refs/remotes/origin/HEAD
DEBU Checked out Git ref=refs/remotes/origin/HEAD
DEBU Current HEAD="cb8d0695f7bd445101b40c HEAD"
DEBU Current BASE="91a4a3b471ea17915b40cd2b HEAD"PePe Amengual7 months ago
how do I null a value with YQ expressions ?
!terraform.output functionappwindows/pdf "function_app_rg_name // """""