#atmos - June 2022

huge shoutout to @Andriy Knysh (Cloud Posse), great help and awesome experience doing a PR for atmos

v1.4.19
what
add the processing of ENV vars to atmos workflow command
why
take into account ATMOS_WORKFLOWS_BASE_PATH ENV var
While all steps in a workflow processed the ENV vars, the atmos workflow command did not, and ATMOS_WORKFLOWS_BASE_PATH ENV var was not used

Does anyone mind explaining again the purpose of /catalog/ set of YAML configs and what does the tool internally do with this?
https://github.com/cloudposse/atmos/tree/master/examples/complete/stacks/catalog

Also, are you manually creating all those YAML files, or are you using some tooling to generate them from TF variables files ?

v1.4.19
what
add the processing of ENV vars to atmos workflow command
why
take into account ATMOS_WORKFLOWS_BASE_PATH ENV var
While all steps in a workflow processed the ENV vars, the atmos workflow command did not, and ATMOS_WORKFLOWS_BASE_PATH ENV var was not used

@Andriy Knysh (Cloud Posse), is there a way for tool to generate entire proposed directories structure in any directory in which it is run ?

I have been able to run atmos terraform init and atmos terraform plan but it still uses local state files - it doesn't start using s3 backend for state and DynamoDB.

I am not sure what I am doing wrong ...

In my components/terraform/iam/user/backend.tf I have ..

terraform {
  backend "s3" {
    # Filled out by atmos from stacks/globals/globals.yaml
  }
}

In my stacks/globals/globals.yaml I have ...

terraform:
  vars: {}

  backend_type: s3 # s3, remote, vault, static, azurerm, etc.
  backend:
    s3:
      encrypt: true
      bucket: "<REDACTED>"
      key: "terraform.tfstate"
      dynamodb_table: "<REDACTED>"
      acl: "bucket-owner-full-control"
      region: "us-west-2"
      role_arn: null

...

When I run atmos terraform init iam/user -s nbi-ops-uw2-devops ... it respects backend configuration in backend.tf file but it is not being fed those backend-related variables from atmos... so it prompts me to enter values ...

I feel like the example is missing this piece ....

I see also that majority of Cloudposse root modules (listed in terraform registry) are not defining any partial backends ...

Re: @azec missing backend 🧵

v1.4.20
what
Update Terraform workspace calculation for legacy Spacelift stack processor
why
LegacyTransformStackConfigToSpaceliftStacks function in the Spacelift stack processor was used to transform the infrastructure stacks to Spacelift stacks using legacy code (and old versions of terraform-yaml-stack-config module) that does not take into account atmos.yaml CLI config - this is very old code that does not know anything about atmos CLI config and it was maintained to support the old versions of...

dumb question: How do you force-unlock with atmos?
atmos terraform force-unlock ID -s <stack>
ID becomes the component and fails.

v1.4.21
what
Update atmos docs and GitHub workflows
Use GitHub environments for deployment
Upgrade to Go version 1.18
why
New CLI documentation sections (describe all CLI commands that atmos supports)
Use GitHub environments for deployment to take advantage of GitHub deployment API and UI (and not comment on PR with deployment URL to not pollute the PR with unnecessary comments)
Go version 1.18 supports many new features including generics and allowing using any keyword instead of interface{} which...

v1.4.21
what
Update atmos docs and GitHub workflows
Use GitHub environments for deployment
Upgrade to Go version 1.18
why
New CLI documentation sections (describe all CLI commands that atmos supports)
Use GitHub environments for deployment to take advantage of GitHub deployment API and UI (and not comment on PR with deployment URL to not pollute the PR with unnecessary comments)
Go version 1.18 supports many new features including generics and allowing using any keyword instead of interface{} which...

Hey everyone — I’m doing some catch up on Atmos and I have a question:
1, Would you recommend using Atmos for creating a reference architect (in AWS) such as the AWS Org?

Hey @Andriy Knysh (Cloud Posse) — Running into an spacelift / stacks config error and wondering if you’ve seen it before or can point me in the right direction. I just quickly peeled back all the layers and I’m at the point that I would want to crack open the provider / atmos to get more debug information from the golang code, but I of course don’t want to do that 😁

Hi everyone 🙂 I'm struggling to set the AWS profile correctly here. I've looked through the docs and the Github repo and it's still not clear what I'm missing. Thanks in advance for the help!

atmos helmfile template aws-load-balancer-controller --stack=uw2-sandbox

I have a uw2-sandbox.yaml file with this component:

  helmfile:
    aws-load-balancer-controller:
      vars:
        installed: true

and the profile is being set to an unexpected value, causing the update-kubeconfig command to fail:

Variables for the component 'aws-load-balancer-controller' in the stack 'uw2-sandbox':

environment: uw2
installed: true
region: us-west-2
stage: sandbox

Using AWS_PROFILE=--gbl-sandbox-helm

/usr/local/bin/aws --profile --gbl-sandbox-helm eks update-kubeconfig --name=--uw2-sandbox--eks-cluster --region=us-west-2 --kubeconfig=/dev/shm/uw2-sandbox-kubecfg

aws: error: argument --profile: expected one argument

I have a feeling it's tenant/namespace/etc weirdness since I'm not using a tenant and it looks like the profile and name values are missing some interpolated string

Hello I’m planning on using Atmos in a test deploy (to learn) .

I’m reading through the accounts module https://github.com/cloudposse/terraform-aws-components/tree/master/modules/account
Just to confirm using a dash - for the account names are not permitted?

Example:

components:
  terraform:
    account:
      backend:
        s3:
          role_arn: null
      vars:
        enabled: true
        account_email_format: aws+lops-%s@lunarops.io
        account_iam_user_access_to_billing: DENY
        organization_enabled: true
        aws_service_access_principals:
          - <http://cloudtrail.amazonaws.com|cloudtrail.amazonaws.com>
          - <http://guardduty.amazonaws.com|guardduty.amazonaws.com>
          - <http://ipam.amazonaws.com|ipam.amazonaws.com>
          - <http://ram.amazonaws.com|ram.amazonaws.com>
          - <http://securityhub.amazonaws.com|securityhub.amazonaws.com>
          - <http://servicequotas.amazonaws.com|servicequotas.amazonaws.com>
          - <http://sso.amazonaws.com|sso.amazonaws.com>
          - <http://securityhub.amazonaws.com|securityhub.amazonaws.com>
          - <http://auditmanager.amazonaws.com|auditmanager.amazonaws.com>
        enabled_policy_types:
          - SERVICE_CONTROL_POLICY
          - TAG_POLICY
        organization_config:
          root_account:
            name: core-root
            stage: root
            tags:
              eks: false
          accounts: []
          organization:
            service_control_policies:
              - DenyNonNitroInstances
          organizational_units:
            - name: core
              accounts:
                - name: core-artifacts
                  tenant: core
                  stage: artifacts
                  tags:
                    eks: false
                - name: core-audit
                  tenant: core
                  stage: audit
                  tags:
                    eks: false
                - name: core-auto
                  tenant: core
                  stage: auto
                  tags:
                    eks: true
                - name: core-corp
                  tenant: core
                  stage: corp
                  tags:
                    eks: true
                - name: core-dns
                  tenant: core
                  stage: dns
                  tags:
                    eks: false
                - name: core-identity
                  tenant: core
                  stage: identity
                  tags:
                    eks: false
                - name: core-demo
                  tenant: core
                  stage: demo
                  tags:
                    eks: false
                - name: core-network
                  tenant: core
                  stage: network
                  tags:
                    eks: false
                - name: core-public
                  tenant: core
                  stage: public
                  tags:
                    eks: false
                - name: core-security
                  tenant: core
                  stage: security
                  tags:
                    eks: false
              service_control_policies:
                - DenyLeavingOrganization
            - name: plat
              accounts:
                - name: plat-dev
                  tenant: plat
                  stage: dev
                  tags:
                    eks: true
                - name: plat-sandbox
                  tenant: plat
                  stage: sandbox
                  tags:
                    eks: true
                - name: plat-staging
                  tenant: plat
                  stage: staging
                  tags:
                    eks: true
                - name: plat-prod
                  tenant: plat
                  stage: prod
                  tags:
                    eks: true
              service_control_policies:
                - DenyLeavingOrganization

Hello again 👋 I'm struggling to use aws-vault with atmos because updating the atmos eks update-kubeconfig command uses --profile, which isn't playing nicely with assuming a role through aws-vault that requires 2FA.

I know CloudPosse has moved on to using Leapp so I'm going to give that a try. In the meantime, is there anything obvious I might be missing to get aws-vault to play more nicely with atmos?

Hello yet again 👋 I encountered some unexpected behavior (and a misleading error message) with creating a Terraform component that has a variable named environment.

I'm planning on switching to using CloudPosse's region/namespace/stage nomenclature soon, but didn't expect this to fail in the meantime. Clearly there's some variable shadowing going on. I can work around it, but wanted to paste it here anyway - and thanks for building such an awesome tool!

# uw2-sandbox.yaml

components:
  terraform:
    eks-iam:
      backend:
        s3:
          workspace_key_prefix: "eks-iam"
      vars:
        environment: "sandbox"

And the output:

$ atmos terraform plan eks-iam --stack=uw2-sandbox

Searched all stack files, but could not find config for the component 'eks-iam' in the stack 'uw2-sandbox'.
Check that all attributes in the stack name pattern '{environment}-{stage}' are defined in the stack config files.
Are the component and stack names correct? Did you forget an import?

Hi there!
I've been using atmos now for 1 month successfully form my workstation to decouple Terraform modules from config parameters/variables.
I am getting ready to try this within GitLab CI/CD.
I am curious whether you have any recommendations or examples (even if they are from different CI/CD system, e.g. GitHub Actions) on how you work with atmos from CI/CD pipelines?
Do you use things like GNU Make for each infra repo with tasks that use atmos , or something else?
I have non-root modules Terraform repository and for now just 1 repository with live infra (similar to what CloudPosse presented in office hours multiple times). My live repository is broken down to folders for each AWS Account and each of those top-level folders then has atmos -suggested structure.

v1.4.22
what
Add ATMOS_CLI_CONFIG_PATH ENV var
Detect more YAML stack misconfigurations
Add functionality to define atmos custom CLI commands
why

ATMOS_CLI_CONFIG_PATH ENV var allows specifying the location of atmos.yaml CLI config file. This is useful for CI/CD environments (e.g. Spacelift) where an infrastructure repository gets loaded into a custom path and atmos.yaml is not in the locations where atmos expects to find it (no need to copy atmos.yaml into /usr/local/etc/atmos/atmos.yaml)


Detect...