22 messages
👽️
Josh Hollowayalmost 4 years ago
So I've been thinking about how Atmos plays with larger infra deployments (read multiple accounts / envs).
Typically for AWS, most people have several accounts across their org:
• management account (e.g. root)
• identity
• audit
• infrastructure
◦ dev
◦ prd
• <org_unit> (e.g. SaaS)
◦ <team> (e.g. payments)
▪︎ dev
▪︎ prd
So far, I've had the
e.g.
But also wondered what others have done so far / might suggest.
Typically for AWS, most people have several accounts across their org:
• management account (e.g. root)
• identity
• audit
• infrastructure
◦ dev
◦ prd
• <org_unit> (e.g. SaaS)
◦ <team> (e.g. payments)
▪︎ dev
▪︎ prd
So far, I've had the
stacks/ dir mirror this layout and then been using tenant to ensure unique statefile paths (e.g. tenant: saas-payments )e.g.
stacks/
gbl-root.yaml
gbl-identity.yaml
gbl-audit.yaml
infra/
gbl-dev.yaml
ue1-dev.yaml
gbl-prd.yaml
saas/
payments/
gbl-dev.yamlBut also wondered what others have done so far / might suggest.
rssalmost 4 years ago(edited)
v1.4.1
what
Add settings.spacelift.stack_name_pattern
Fix parsing YAML config and detection of stacks when the stack name (on the command line) is the same as the YAML config file name in a subfolder
why
settings.spacelift.stack_name_pattern allows overriding Spacelift stack names. Supported tokens: {namespace}, {tenant}, {environment}, {stage}, {component}
components:
terraform:
"test/test-component-override-2":
settings:
spacelift:
workspace_enabled: true
#...
what
Add settings.spacelift.stack_name_pattern
Fix parsing YAML config and detection of stacks when the stack name (on the command line) is the same as the YAML config file name in a subfolder
why
settings.spacelift.stack_name_pattern allows overriding Spacelift stack names. Supported tokens: {namespace}, {tenant}, {environment}, {stage}, {component}
components:
terraform:
"test/test-component-override-2":
settings:
spacelift:
workspace_enabled: true
#...
rssalmost 4 years ago(edited)
v1.4.2
what
Add init_run_reconfigure CLI config
Update stack_name_pattern
Disable running terraform plan and terraform workspace on abstract components
why
init_run_reconfigure CLI config allows enabling/disabling the -reconfigure argument for terraform init when running it before running other terraform commands
Don't use the default stack_name_pattern because it used {tenant} which is not available for all clients
Running terraform plan and terraform workspace on abstract components creates...
what
Add init_run_reconfigure CLI config
Update stack_name_pattern
Disable running terraform plan and terraform workspace on abstract components
why
init_run_reconfigure CLI config allows enabling/disabling the -reconfigure argument for terraform init when running it before running other terraform commands
Don't use the default stack_name_pattern because it used {tenant} which is not available for all clients
Running terraform plan and terraform workspace on abstract components creates...
rssalmost 4 years ago
v1.4.2
what
Add init_run_reconfigure CLI config
Update stack_name_pattern
Disable running terraform plan and terraform workspace on abstract components
why
init_run_reconfigure CLI config allows enabling/disabling the -reconfigure argument for terraform init when running it before running other terraform commands
Don't use the default stack_name_pattern because it used {tenant} which is not available for all clients
Running terraform plan and terraform workspace on abstract components creates...
what
Add init_run_reconfigure CLI config
Update stack_name_pattern
Disable running terraform plan and terraform workspace on abstract components
why
init_run_reconfigure CLI config allows enabling/disabling the -reconfigure argument for terraform init when running it before running other terraform commands
Don't use the default stack_name_pattern because it used {tenant} which is not available for all clients
Running terraform plan and terraform workspace on abstract components creates...
Zach Bridgesalmost 4 years ago
hey yall! hope everyone is doing well. I have run into problems with atmos reading really large templates. i tried to read through the code to see if i could fix it but atmos v0.22 is written in variant and is not that amenable to being changed (the problem lies in a dependency’s dependency).
as a workaround I am wondering if there is an alternative in recent versions of atmos to the atmos v0.22 command
as a workaround I am wondering if there is an alternative in recent versions of atmos to the atmos v0.22 command
atmos stack config --config-type all it looks like you can get the config per component but not per stack.rssalmost 4 years ago(edited)
v1.4.3
what
Add --dry-run command-line flag to all commands and workflows
why
Helps debugging atmos commands and workflows
The --dry-run flag shows all the flow and commands without executing them and without writing files to the file system (e.g. varfiles and backend config are not written)
The --dry-run flag shows all the workflow steps without executing them
test
atmos terraform plan test/test-component-override -s=tenant1-ue2-dev --dry-run
Variables for the component...
what
Add --dry-run command-line flag to all commands and workflows
why
Helps debugging atmos commands and workflows
The --dry-run flag shows all the flow and commands without executing them and without writing files to the file system (e.g. varfiles and backend config are not written)
The --dry-run flag shows all the workflow steps without executing them
test
atmos terraform plan test/test-component-override -s=tenant1-ue2-dev --dry-run
Variables for the component...
rssalmost 4 years ago(edited)
v1.4.4
what
Add atmos describe stacks command
Allow writing the result to a file by using --file command-line flag
Allow formatting the result as YAML or JSON by using --format command-line flag
Allow filtering of the result by using the command-line flags: stack, components-types, components, sections
Available component sections: backend, backend_type, deps, env, inheritance, metadata, remote_state_backend, remote_state_backend_type, settings, vars
why
Command to show stack configs and all the...
what
Add atmos describe stacks command
Allow writing the result to a file by using --file command-line flag
Allow formatting the result as YAML or JSON by using --format command-line flag
Allow filtering of the result by using the command-line flags: stack, components-types, components, sections
Available component sections: backend, backend_type, deps, env, inheritance, metadata, remote_state_backend, remote_state_backend_type, settings, vars
why
Command to show stack configs and all the...
Erik Osterman (Cloud Posse)almost 4 years ago
@here ☝️ this is awesome! you can now query your stack configurations
rssalmost 4 years ago(edited)
v1.4.4
what
Add atmos describe stacks command
Allow writing the result to a file by using --file command-line flag
Allow formatting the result as YAML or JSON by using --format command-line flag
Allow filtering of the result by using the command-line flags: stack, component-types, components, sections
Available component sections: backend, backend_type, deps, env, inheritance, metadata, remote_state_backend, remote_state_backend_type, settings, vars
why
Command to show stack configs and all the...
what
Add atmos describe stacks command
Allow writing the result to a file by using --file command-line flag
Allow formatting the result as YAML or JSON by using --format command-line flag
Allow filtering of the result by using the command-line flags: stack, component-types, components, sections
Available component sections: backend, backend_type, deps, env, inheritance, metadata, remote_state_backend, remote_state_backend_type, settings, vars
why
Command to show stack configs and all the...
Michael Dizonalmost 4 years ago
is there a way to create state information for resources (Like an aws org account) that already exist (and may have been created outside of TF)?
rssalmost 4 years ago(edited)
v1.4.5
what
Fix detection of component dependencies for imported YAML config files
why
If a file was imported and did not contain a vars section, global or related to the component, the imported file was not included in the component dependencies (deps section), and a Spacelift label for the file was not created (if the imported file changes, Spacelift would not notice the change and would not run the stack)
Update the dependencies logic to check for these sections: backend, backend_type, env,...
what
Fix detection of component dependencies for imported YAML config files
why
If a file was imported and did not contain a vars section, global or related to the component, the imported file was not included in the component dependencies (deps section), and a Spacelift label for the file was not created (if the imported file changes, Spacelift would not notice the change and would not run the stack)
Update the dependencies logic to check for these sections: backend, backend_type, env,...
rssalmost 4 years ago(edited)
v1.4.6
what
If auto_generate_backend_file is true (we are auto-generating backend files), remove backend.tf.json when executing atmos terraform clean command
why
Useful when using different backends for the same component in different stacks
what
If auto_generate_backend_file is true (we are auto-generating backend files), remove backend.tf.json when executing atmos terraform clean command
why
Useful when using different backends for the same component in different stacks
rssalmost 4 years ago(edited)
v1.4.7
what
Take into account init_run_reconfigure CLI config and --init-run-reconfigure command-line argument when running atmos terraform init ... command
why
atmos terraform init must behave the same as all other commands that use init_run_reconfigure CLI config and --init-run-reconfigure command-line argument
what
Take into account init_run_reconfigure CLI config and --init-run-reconfigure command-line argument when running atmos terraform init ... command
why
atmos terraform init must behave the same as all other commands that use init_run_reconfigure CLI config and --init-run-reconfigure command-line argument
rssalmost 4 years ago(edited)
v1.4.8
what
Add metadata.terraform_workspace_pattern
why
We already have metadata.terraform_workspace using which we can specify/override the terraform workspace for a component
metadata.terraform_workspace_pattern introduces a pattern to override a terraform workspace for a component
metadata:
# Override Terraform workspace
terraform_workspace: xxxxxxxxxxxxxxx
terraform_workspace_pattern: "{tenant}-{environment}-{stage}-{component}"
The following tokens are supported...
what
Add metadata.terraform_workspace_pattern
why
We already have metadata.terraform_workspace using which we can specify/override the terraform workspace for a component
metadata.terraform_workspace_pattern introduces a pattern to override a terraform workspace for a component
metadata:
# Override Terraform workspace
terraform_workspace: xxxxxxxxxxxxxxx
terraform_workspace_pattern: "{tenant}-{environment}-{stage}-{component}"
The following tokens are supported...
rssalmost 4 years ago(edited)
v1.4.9
what
Add atmos aws commands
Add atmos aws eks update-kubeconfig command
why
Execute aws CLI commands using atmos context (atmos.yaml CLI config and component/stack configurations)
Downloading kubeconfig from an EKS cluster using `atmos
Allow using this functionality in Terraform by implementing a new data source in the terraform-provider-utils provider (this will be added in a PR in <a...
what
Add atmos aws commands
Add atmos aws eks update-kubeconfig command
why
Execute aws CLI commands using atmos context (atmos.yaml CLI config and component/stack configurations)
Downloading kubeconfig from an EKS cluster using `atmos
Allow using this functionality in Terraform by implementing a new data source in the terraform-provider-utils provider (this will be added in a PR in <a...
rssalmost 4 years ago(edited)
v1.4.10
what
If functions that are used by Terraform providers throw errors, print errors to std.Error
why
Terraform providers only see errors that are sent to std.Error
what
If functions that are used by Terraform providers throw errors, print errors to std.Error
why
Terraform providers only see errors that are sent to std.Error
Matt Gowiealmost 4 years ago
Another Atmos in the terraform space — https://github.com/simplygenius/atmos
Just heard about it on a call with a company. Seems fairly dead and not all the interesting of a project, but it is funny that name would get chosen twice in the TF space.
Just heard about it on a call with a company. Seems fairly dead and not all the interesting of a project, but it is funny that name would get chosen twice in the TF space.
Erik Osterman (Cloud Posse)almost 4 years ago
Why we chose atmos name: https://www.space.com/terraforming-in-alien-universe
rssalmost 4 years ago(edited)
v1.4.11
what
Support dashes - in the tenant, environment and stage names
In the examples, add a new stage test-1 and add tests for components, stacks, and spacelift to test having a dash in the stage name (the file name itself being without dashes)
why
The old atmos supported it (because it was filename-based, not logical stack name based)
Some clients want to name tenants/environment/stages with dashes in the names (and some already have it, so we need to support that when converting from the old to...
what
Support dashes - in the tenant, environment and stage names
In the examples, add a new stage test-1 and add tests for components, stacks, and spacelift to test having a dash in the stage name (the file name itself being without dashes)
why
The old atmos supported it (because it was filename-based, not logical stack name based)
Some clients want to name tenants/environment/stages with dashes in the names (and some already have it, so we need to support that when converting from the old to...
rssalmost 4 years ago(edited)
v1.4.3
what
Add --dry-run command-line flag to all commands and workflows
why
Helps debugging atmos commands and workflows
The --dry-run flag shows all the flow and commands without executing them and without writing files to the file system (e.g. varfiles and backend config are not written)
The --dry-run flag shows all the workflow steps without executing them
tests and examples
Click to expandatmos terraform plan test/test-component-override -s=tenant1-ue2-dev --dry-run
Variables for the...
what
Add --dry-run command-line flag to all commands and workflows
why
Helps debugging atmos commands and workflows
The --dry-run flag shows all the flow and commands without executing them and without writing files to the file system (e.g. varfiles and backend config are not written)
The --dry-run flag shows all the workflow steps without executing them
tests and examples
Click to expandatmos terraform plan test/test-component-override -s=tenant1-ue2-dev --dry-run
Variables for the...
rssalmost 4 years ago(edited)
v1.4.4
what
Add atmos describe stacks command
Allow writing the result to a file by using --file command-line flag
Allow formatting the result as YAML or JSON by using --format command-line flag
Allow filtering of the result by using the command-line flags: stack, component-types, components, sections
Available component sections: backend, backend_type, deps, env, inheritance, metadata, remote_state_backend, remote_state_backend_type, settings, vars
why
Command to show stack configs and all the...
what
Add atmos describe stacks command
Allow writing the result to a file by using --file command-line flag
Allow formatting the result as YAML or JSON by using --format command-line flag
Allow filtering of the result by using the command-line flags: stack, component-types, components, sections
Available component sections: backend, backend_type, deps, env, inheritance, metadata, remote_state_backend, remote_state_backend_type, settings, vars
why
Command to show stack configs and all the...
Chris Bloemkeralmost 4 years ago
Good morning all! We are trying to add a new mono repo for our SRE team to Spacelift with Atmos and it seems like Spacelift can’t find the stacks based on
Spacelift run error:
Our directory structure is the same as the atmos example here where we have at the root of the repo:
Trying to see if there is another variable overriding
base_path$ atmos describe config | jq '.Components.Terraform.base_path'
"/components/terraform"
$ atmos describe config | jq '.Stacks.base_path'
"/stacks"Spacelift run error:
No stack config files found in the provided paths:
│ - /mnt/workspace/source/components/terraform/spacelift/stacks/**/*Our directory structure is the same as the atmos example here where we have at the root of the repo:
components
stacks
atmos.yamlTrying to see if there is another variable overriding
stacks.base_path somewhere else.