29 messages
π½οΈ
Hans Dover 2 years ago
anybody already got some renovatebot config to perform component updates?
Hans Dover 2 years ago
and is there a way to suppress the output of the init (not skipping the init), eg when doing a
atmos terraform show -json comp -s stack ?Jonasover 2 years ago
Is there some built-in way to automatically wrap the atmos command with
ATMOS_BASE_PATH ATMOS_CLI_CONFIG_PATH and setting it to PWD in order to use remote module states when working in the current dir, without using geodesic or moving the atmos file? Iβm thinking makefile probably would be able to do this, but just wondering if thereβs any other solutionsjonjitsuover 2 years ago
Are there any minimal examples of an atmos driven repo? As awesome as atmos/examples/complete is it's a lot when first approaching something new. What would be the absolute minimal repo which deploys a single component in a single stack?
shmileeeover 2 years ago
Hey guys, I'm following
atmos tutorial here and currently stuck on atmos vendor pull --component infra/vpc, keep getting:
No stack config files found in the provided paths:
- /<omitted>/atmos-poc/stacks/orgs/**/*
Check if `base_path`, 'stacks.base_path', 'stacks.included_paths' and 'stacks.excluded_paths' are correctly set in CLI config files or ENV vars.Jonasover 2 years ago
There is a lot of information related to the usage and way of thinking when building with atmos. The components also provide a nice way to better understand the opinionated concepts related to separating and deploying/destroying parts that relate to eachother. However, one important aspect I have failed to discover yet, is how to handle EKS service roles for the application services you are building. These service roles will all have some kind of dependence on other components of the infrastructure. Whats the best practise setting up something like that? Do you just write something custom for it and reference the arns from the remote state?
kevcubeover 2 years ago
I think
it should insted default to
atmos vendor pull should not default to atmos vendor pull [STACK] when it will always give the error command 'atmos vendor pull --stack <stack>' is not implemented yetit should insted default to
atmos vendor pull [COMPONENT]johncblandiiover 2 years ago
since
ex:
β’ dev:
β’ prod:
we would rather not have
just curious of any solutions.
inherits is an array and doesn't merge when overridden, is there a solution where we could inherit without having to copy all other inherits from the main file?ex:
β’ dev:
retention_in_days: 3β’ prod:
retention_in_days: 120we would rather not have
120 sprayed across ~68 files so would like to use inherits. doing so means updating the inheritance in dev and prod each time it changes (surely someone will forget and break prod).just curious of any solutions.
Patrick McDonaldover 2 years ago
what is this used for?
settings:
config:
is_prod: falseA
Andrew Ochsnerover 2 years ago
Seems Terraform is implementing stacks...
US and foreign patents pending even! https://www.hashicorp.com/blog/new-terraform-testing-and-ux-features-reduce-toil-errors-and-costsAndrew Ochsnerover 2 years ago
I'm about to do a bit of a spike on refactoring from a monolithic, landing zone terraform root module & state file to atmos... any tools or techniques to help w/ this migration... any tips? seems a bit daunting
rssover 2 years ago(edited)
v1.46.0
what
Update to Go 1.21
Update/improve Atmos help for atmos-specific commands
Improve atmos vendor pull command error handling
Add Atmos custom commands to atmos.yaml and docs
Add examples of Rego policies for Atmos components and stacks validation. Update docs. See https://atmos.tools/core-concepts/components/validation/
Add Sprig functions support to Go templates in Atmos vendoring
Improve context...
what
Update to Go 1.21
Update/improve Atmos help for atmos-specific commands
Improve atmos vendor pull command error handling
Add Atmos custom commands to atmos.yaml and docs
Add examples of Rego policies for Atmos components and stacks validation. Update docs. See https://atmos.tools/core-concepts/components/validation/
Add Sprig functions support to Go templates in Atmos vendoring
Improve context...
rssover 2 years ago
v1.46.0
what
Update to Go 1.21
Update/improve Atmos help for atmos-specific commands
Improve atmos vendor pull command error handling
Add Atmos custom commands to atmos.yaml and docs
Add examples of Rego policies for Atmos components and stacks validation. Update docs. See https://atmos.tools/core-concepts/components/validation/
Add Sprig functions support to Go templates in Atmos vendoring
Improve context...
what
Update to Go 1.21
Update/improve Atmos help for atmos-specific commands
Improve atmos vendor pull command error handling
Add Atmos custom commands to atmos.yaml and docs
Add examples of Rego policies for Atmos components and stacks validation. Update docs. See https://atmos.tools/core-concepts/components/validation/
Add Sprig functions support to Go templates in Atmos vendoring
Improve context...
kevcubeover 2 years ago
It'd be nice to have developer tooling for atmos like a yaml spec in VScode for stack files
Jonasover 2 years ago(edited)
If I want to make a separate environment in a different continent/account, what would the best property of the label component be to change to differentiate between the 2? Having a hard time choosing as it impacts other parts of the stack too. Ideally I think I would be
tenant but then I also would need to have tenant in the name_pattern config for atmos. But this would break all of the previous stacks. For example, if I want to use something like aws-team-roles, if both continents have production environments they would both be in stack acme-gbl-prod ({namespace}-{environment}-{stage}) ,which would not work. So then naming them differently would be the only option.Matthew Regglerover 2 years ago(edited)
Are there downsides to directly modifying the
Example:
I think the former approach could lead to a very flat S3 backend with potentially hundreds of top-level directories, with local sub-folders under components only distinguishable by prefix.
The second approach requires modifying the catalog of each new component to override the autogenerated workspace prefix that collapses all
a) mirrors the local component directory structure
b) groups all backend state files by their directory, whether this be a project, a team, service or βkindβ
Iβve tested this approach in vanilla terraform, as there are some old unresolved github issues where people attempting this unsuccessfully, (resorting to hacking the
Will this cause any problems with Atmos under the hood?
workspace_key_prefix to force the S3 backend structure to match a non-standard component hierarchy?Example:
# local directory structure
components/
terraform/
account/
account-map/
vpc/
bespoke/
component-a/
component-b/
# auto-generated workspaces+prefixes in S3
account/core-ue1-root/terraform.tfstate
account-map/core-ue1-root/terraform.tfstate
...
bespoke-component-a/plat-ue2-devt/terraform.tfstate
bespoke-component-a/plat-ue2-prod/terraform.tfstate
...
# post hacking the workspace prefix to "bespoke/<component-name>"
account/core-ue1-root/terraform.tfstate
account-map/core-ue1-root/terraform.tfstate
...
bespoke/component-a/plat-ue2-devt/terraform.tfstate
bespoke/component-a/plat-ue2-prod/terraform.tfstate
...I think the former approach could lead to a very flat S3 backend with potentially hundreds of top-level directories, with local sub-folders under components only distinguishable by prefix.
The second approach requires modifying the catalog of each new component to override the autogenerated workspace prefix that collapses all
/ characters to - , but would lead to a backend that:a) mirrors the local component directory structure
b) groups all backend state files by their directory, whether this be a project, a team, service or βkindβ
Iβve tested this approach in vanilla terraform, as there are some old unresolved github issues where people attempting this unsuccessfully, (resorting to hacking the
key property), but it does appear to work in modern TerraformWill this cause any problems with Atmos under the hood?
Jonasover 2 years ago
Is there a way to unset a deep merged property in atmos? I need certain tags on all my resources so somewhere in the hierarchy, its included in the defaults. But for kubernetes resources these tags are not needed and/or error. Setting these to
null gives me another error. The only other solution I see would be to be more selective in importing these default tags, but this means more imports, so rather want to βexcludeβ rather than βincludeβ.Hans Dover 2 years ago(edited)
For the atmos workflow files, it would be a nice addition to be able to tag the upper level steps, and be able to run all these steps specifying that tag (or skip), eg:
(bit like ansible playbooks)
workflows:
main-step-1:
tags: [ slow ]
steps: [ ... ]
main-step-2:
tags: [ fast ]
steps: [ ... ]
main-step-3:
tags: [ slow ]
steps: [ ... ]atmos workflow --use-tags fast -f dummy or atmos workflow --skip-tags slow -f dummy(bit like ansible playbooks)
RBover 2 years ago
I remember when running a terraform plan/apply, atmos would show the deep merge of the yaml stack component at the very top. I no longer see it and cannot find a way to turn it back on.
Hans Dover 2 years ago
For atmos and subcommand, I see i can define a flag of type
bool. How can I access this flag in the template, eg {{ .Flags.<flagname> }} throws atemplate: step-0:5:14: executing "step-0" at <.Flags.xroot>: map has no entry for key "xroot"Hans Dover 2 years ago
For atmos and subcommand, when a flag is like
require-root , how can I access this flag in the template?RBover 2 years ago(edited)
Quick question, has atmos ever had a 3rd party security audit? asking for a client that has high security requirements
rssover 2 years ago(edited)
v1.47.0
what
Add vendor.yaml as a new way of vendoring components, stacks and other artifacts
Add/update docs
https://atmos.tools/core-concepts/vendoring/
https://atmos.tools/cli/commands/vendor/pull/
why
Atmos natively supports the concept of "vendoring", which is making copies of the 3rd party components, stacks, and other artifacts in your own...
what
Add vendor.yaml as a new way of vendoring components, stacks and other artifacts
Add/update docs
https://atmos.tools/core-concepts/vendoring/
https://atmos.tools/cli/commands/vendor/pull/
why
Atmos natively supports the concept of "vendoring", which is making copies of the 3rd party components, stacks, and other artifacts in your own...
rssover 2 years ago
v1.47.0
what
Add vendor.yaml as a new way of vendoring components, stacks and other artifacts
Add/update docs
https://atmos.tools/core-concepts/vendoring/
https://atmos.tools/cli/commands/vendor/pull/
why
Atmos natively supports the concept of "vendoring", which is making copies of the 3rd party components, stacks, and other artifacts in your own...
what
Add vendor.yaml as a new way of vendoring components, stacks and other artifacts
Add/update docs
https://atmos.tools/core-concepts/vendoring/
https://atmos.tools/cli/commands/vendor/pull/
why
Atmos natively supports the concept of "vendoring", which is making copies of the 3rd party components, stacks, and other artifacts in your own...
Nateover 2 years ago
Hi there! New user of Atmos here. Just started experimenting yesterday and so far I like the promise of the framework/tool. I created a few simple stacks to experiment and am stuck with a terraform state related issue. I initially followed the guidance here (https://docs.cloudposse.com/components/library/aws/account/) to create a set of AWS accounts in a brand new AWS Master account (no other resource created). I am now trying to move the local state to S3 and am getting the following error:
Error: Error loading state error
β
β with module.assume_role["hct-gbl-root-tfstate-purple-cow"].module.allowed_role_map.module.account_map.data.terraform_remote_state.data_source[0],
β on .terraform/modules/assume_role.allowed_role_map.account_map/modules/remote-state/data-source.tf line 91, in data "terraform_remote_state" "data_source":
β 91: backend = local.ds_backend
β
β error loading the remote state: S3 bucket does not exist.
β
β The referenced S3 bucket must have been previously created. If the S3 bucket
β was created within the last minute, please wait for a minute or two and try
β again.
β
β Error: NoSuchBucket: The specified bucket does not exist
β status code: 404, request id: W7M341334K7N0DVR, host id: mWPCc2QiX7Aw3ar+uB1wravpTivYZdCnVqVPoNaQXe7NNyOdKGyHbNcR5hJgZ0R5MoYY5TvwN+M=Nateover 2 years ago
My understanding is that the S3 bucket gets created as part of the execution of this TF script. What am I missing?
Nateover 2 years ago
Command executed:
atmos terraform plan tfstate-backend --stack core-ue1-rootNateover 2 years ago
Any help will be greatly appreciated!
Nateover 2 years ago
I am getting the following error while provisioning a simple "vpc-flow-logs-bucket" component as outlined in the example.
It looks like the iam-roles module (in account_map_ is looking for accounts using just the stage attribute where as the accounts created follow the format "tenant-stage". The list of accounts from account_map module has an account called "plat-dev", but the tenant name is not being included in the look up above. I did some more digging and it appears others had similar issue and they resolved it by passing an updated "description_format" variable to the stack.
I did include the above block in my stack def - but nothing seems to be working. Did anyone else face a similar issue and if you did, what was the solution?
Error: Invalid index
β
β on ../account-map/modules/iam-roles/main.tf line 38, in locals:
β 38: local.account_map.full_account_map[local.account_name]
β βββββββββββββββββ
β β local.account_map.full_account_map is object with 12 attributes
β β local.account_name is "dev"
β
β The given key does not identify an element in this collection value.It looks like the iam-roles module (in account_map_ is looking for accounts using just the stage attribute where as the accounts created follow the format "tenant-stage". The list of accounts from account_map module has an account called "plat-dev", but the tenant name is not being included in the look up above. I did some more digging and it appears others had similar issue and they resolved it by passing an updated "description_format" variable to the stack.
descriptor_formats:
account_name:
format: "%v-%v"
labels:
- tenant
- stage
stack:
format: "%v-%v-%v"
labels:
- tenant
- environment
- stageI did include the above block in my stack def - but nothing seems to be working. Did anyone else face a similar issue and if you did, what was the solution?