#terragrunt - February 2019

terragruntArchived

92 messages

Terragrunt discussions
Archive: https://archive.sweetops.com/terragrunt/

February 2019

Samulialmost 8 years ago

any best practices on how to reference between terraform dependencies on different terragrunt modules. eg. I have vpc as a module and bastion as another module (terragrunt modules both) and I need to reference the vpc-id (and subnet-ids) from vpc in bastion

Erik Osterman (Cloud Posse)almost 8 years ago

yes, use SSM

Erik Osterman (Cloud Posse)almost 8 years ago

https://www.terraform.io/docs/providers/aws/r/ssm_parameter.html

Erik Osterman (Cloud Posse)almost 8 years ago

or use remote state provider

Erik Osterman (Cloud Posse)almost 8 years ago

https://www.terraform.io/docs/providers/terraform/d/remote_state.html

Erik Osterman (Cloud Posse)almost 8 years ago

we have examples of both in this repo https://github.com/cloudposse/terraform-root-modules

joshmyersalmost 8 years ago

@Samuli Are these modules being run in the same Terraform run? (do they share state)

Samulialmost 8 years ago

It would be straight forward if they did but with terragrunt modules they are not. So I went with remote_state..

erikover 7 years ago

set the channel description: Terragrunt discussions
Archive: https://archive.sweetops.com/terragrunt/

Rohit Vermaover 7 years ago

@Erik Osterman (Cloud Posse) how are we suppose to use terragrunt in your new setup. I see eligible terraform.tfvars only in atlantis-repo sub-module.

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/root.cloudposse.co/blob/master/conf/users/terraform.tfvars

Erik Osterman (Cloud Posse)over 7 years ago

this is all you need

Erik Osterman (Cloud Posse)over 7 years ago

that said, I see no strong use-case for it anymore

Erik Osterman (Cloud Posse)over 7 years ago

I think make is a better task runner

Erik Osterman (Cloud Posse)over 7 years ago

and tfenv let’s use define module imports

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/testing.cloudposse.co/blob/master/conf/ecs/.envrc#L2

Rohit Vermaover 7 years ago

want to avoid tfenv, it seems over populating the environment by appending all current env variables with TF_VAR

Erik Osterman (Cloud Posse)over 7 years ago

with tfenv, we don’t need any wrappers to call terraform

Erik Osterman (Cloud Posse)over 7 years ago

fair enough

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/root.cloudposse.co/blob/master/conf/users/terraform.tfvars#L10-L13

Erik Osterman (Cloud Posse)over 7 years ago

i just don’t like this

Erik Osterman (Cloud Posse)over 7 years ago

overloading .tfvars with a non-portable terraform code that is vendor specific to terragrunt

Rohit Vermaover 7 years ago

also i am thinking may be its kind of simpler if we have a whitelisting variables rather then excluding regex

Erik Osterman (Cloud Posse)over 7 years ago

while envs are universal across apps

erikover 7 years ago

Erik Osterman (Cloud Posse)over 7 years ago

TFENV_WHITELIST is supported

Rohit Vermaover 7 years ago

I saw that, I was saying more about ideology. The whitelist value is .*.

Erik Osterman (Cloud Posse)over 7 years ago

what i don’t like is terraform telling me how envs should look in the first place. just like i don’t like chamber telling me that envs must be upper case (which doesn’t work with terraform)

Erik Osterman (Cloud Posse)over 7 years ago

so we have env warfare

Rohit Vermaover 7 years ago

Should be something like geodesic whitelist, and customer whitelist. Where we convert all the values required for geodesic modules. SPECIALLY the ones which we are currently getting

Erik Osterman (Cloud Posse)over 7 years ago

I just wish terraform and terragrunt would not stipulate a convention on envs and then things would just work. so tfenv is an ambassador. it does the dirty work.

Rohit Vermaover 7 years ago

the approach makes sense in the current context, its just having too much env variables makes my life hard

Erik Osterman (Cloud Posse)over 7 years ago

just set TFENV_WHITELIST and TFENV_BLACKLIST in your environment however you want 😃

Erik Osterman (Cloud Posse)over 7 years ago

we were so busy mapping envs from one tool to the next. that’s what we wanted to avoid.

Rohit Vermaover 7 years ago

too much work to make a module get up. Its as good as doing a export variable in a tfenv.sh

Erik Osterman (Cloud Posse)over 7 years ago

so setting too many envs makes everyones life hard

Erik Osterman (Cloud Posse)over 7 years ago

but the env always has many envs, that’s just a fact-of-linux

Erik Osterman (Cloud Posse)over 7 years ago

on OSX, I have 87 envs that i never set 😃

Erik Osterman (Cloud Posse)over 7 years ago


export|wc

Rohit Vermaover 7 years ago

i have total 57, out of which I know about atleast 40

Erik Osterman (Cloud Posse)over 7 years ago

so in our Dockerfile we had SOOOOOO many envs. it was unmanageable.

Erik Osterman (Cloud Posse)over 7 years ago

so we’ve gotten rid of most of them (compared to before)

Erik Osterman (Cloud Posse)over 7 years ago

then moved to .envrc (direnv) so we localize these settings

Rohit Vermaover 7 years ago

yeah regarding that, I saw below snippet in rc.d terraform

# Translate environment variables to terraform arguments
	[ -z "${TF_FROM_MODULE}" ] || export TF_CLI_INIT_FROM_MODULE="${TF_FROM_MODULE}"
	[ -z "${TF_STATE_FILE}" ] || export TF_CLI_INIT_BACKEND_CONFIG_KEY="${TF_BUCKET_PREFIX}/${TF_STATE_FILE}"
	[ -z "${TF_BUCKET}" ] || export TF_CLI_INIT_BACKEND_CONFIG_BUCKET="${TF_BUCKET}"
	[ -z "${TF_BUCKET_REGION}" ] || export TF_CLI_INIT_BACKEND_CONFIG_REGION="${TF_BUCKET_REGION}"
	[ -z "${TF_DYNAMODB_TABLE}" ] || export TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE="${TF_DYNAMODB_TABLE}"
	[ -z "${AWS_PROFILE}" ] || export TF_CLI_INIT_BACKEND_CONFIG_PROFILE="${AWS_PROFILE}"
	[ -z "${AWS_ROLE_ARN}" ] || export TF_CLI_INIT_BACKEND_CONFIG_ROLE_ARN="${AWS_ROLE_ARN}"

Erik Osterman (Cloud Posse)over 7 years ago

yea, that’s strictly for backwards compatibility

Erik Osterman (Cloud Posse)over 7 years ago

I didn’t want to tell everyone to rewrite their envs 😃

Erik Osterman (Cloud Posse)over 7 years ago

the TF_CLI_* convention is canonical

Rohit Vermaover 7 years ago

which i assume should be good enough if we have all variables in left hand side set

Erik Osterman (Cloud Posse)over 7 years ago(edited)

~~the ones on the LHS are the legacy ones we’ve had in our docs~~

Erik Osterman (Cloud Posse)over 7 years ago

😃

Rohit Vermaover 7 years ago

yeah, my point even if we are using the legacy variables, things should work

Erik Osterman (Cloud Posse)over 7 years ago

I mean the ones in the [ ... ] conditionals

Rohit Vermaover 7 years ago

e.g TF_BUCKET_REGION

Erik Osterman (Cloud Posse)over 7 years ago

yep, so using that mapping they will continue to work

Erik Osterman (Cloud Posse)over 7 years ago

but TF_CLI_INIT_BACKEND_CONFIG_REGION is canonical

Erik Osterman (Cloud Posse)over 7 years ago

in that it maps precisely and consistently to the TF_CLI_ARGS_init=-backend-config=region=blah which is the terraform native convention

Erik Osterman (Cloud Posse)over 7 years ago

technically, even tfenv isn’t needed. it’s just a convenience.

Erik Osterman (Cloud Posse)over 7 years ago

so we can set k/v pairs as ENVs

Erik Osterman (Cloud Posse)over 7 years ago

rather than mucking with the tf flags in compacted envs

Rohit Vermaover 7 years ago

this is what I am doing currently

Erik Osterman (Cloud Posse)over 7 years ago

the compacted envs?

Erik Osterman (Cloud Posse)over 7 years ago(edited)

I don’t like terraform native envs

Erik Osterman (Cloud Posse)over 7 years ago


TF_CLI_ARGS_init="-from-module=git::<https://github.com/cloudposse/terraform-root-modules.git//aws/ecs?ref=tags/0.40.0> -backend-config=region=us-west-2 -backend-config=dynamodb_table=cpco-testing-terraform-state-lock -backend-config=bucket=cpco-testing-terraform-state -backend-config=key=ecs/terraform.tfstate"

Erik Osterman (Cloud Posse)over 7 years ago

that’s what it looks like.

Erik Osterman (Cloud Posse)over 7 years ago(edited)

PIA to toggle individual fields

Erik Osterman (Cloud Posse)over 7 years ago

so we just came up with a convention to not do that. but that’s opinionated and might not suit all parties.

Erik Osterman (Cloud Posse)over 7 years ago

I like it b/c you can set envs at different levels (E.g. Dockerfile, project, parent folder, etc)

Rohit Vermaover 7 years ago

i was doing something like export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=niki-root-terraform-state

Erik Osterman (Cloud Posse)over 7 years ago

that looks good

Rohit Vermaover 7 years ago

but its still asking me for bucket name

Erik Osterman (Cloud Posse)over 7 years ago

TF_CLI_INIT_BACKEND_CONFIG_BUCKET="cpco-testing-terraform-state"

Erik Osterman (Cloud Posse)over 7 years ago

that’s what we are doing

Erik Osterman (Cloud Posse)over 7 years ago(edited)


TF_CLI_PLAN_PARALLELISM=2
TF_CLI_INIT_BACKEND_CONFIG_REGION=us-west-2
TF_CLI_INIT_FROM_MODULE=git::<https://github.com/cloudposse/terraform-root-modules.git//aws/ecs?ref=tags/0.40.0>
TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=cpco-testing-terraform-state-lock
TF_CLI_INIT_BACKEND_CONFIG_BUCKET=cpco-testing-terraform-state
TF_CLI_INIT_BACKEND_CONFIG_KEY=ecs/terraform.tfstate

Erik Osterman (Cloud Posse)over 7 years ago

this definitely works with every terraform * command

Erik Osterman (Cloud Posse)over 7 years ago(edited)

if you use tfenv to cast it to the TF_CLI_ARGS_blah=....

Rohit Vermaover 7 years ago

in short with my legacy variables, use_terraform and terraform init should give me required resuly

Erik Osterman (Cloud Posse)over 7 years ago

true

Erik Osterman (Cloud Posse)over 7 years ago



# Terraform State Bucket
ENV TF_BUCKET_REGION="${AWS_REGION}"
ENV TF_BUCKET="${NAMESPACE}-${STAGE}-terraform-state"
ENV TF_DYNAMODB_TABLE="${NAMESPACE}-${STAGE}-terraform-state-lock"

Rohit Vermaover 7 years ago

i can't use use_terraform also

Erik Osterman (Cloud Posse)over 7 years ago

is what we have in our docker file

Rohit Vermaover 7 years ago

has to source it manually

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/geodesic/blob/master/rootfs/etc/profile.d/direnv.sh

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/geodesic/blob/master/Dockerfile#L220

Rohit Vermaover 7 years ago

something like this is happening

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/geodesic/tree/master/rootfs/etc/direnv

Rohit Vermaover 7 years ago

 ✓  (root-admin) tfstate-backend ⨠  use_terraform 
 ✓  (root-admin) tfstate-backend ⨠  printenv | grep TF_CLI
TF_CLI_INIT_BACKEND_CONFIG_REGION=ap-south-1
TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=niki-root-terraform-state-lock
TF_CLI_INIT_BACKEND_CONFIG_BUCKET=niki-root-terraform-state
TF_CLI_INIT_BACKEND_CONFIG_PROFILE=root-admin
TF_CLI_INIT_BACKEND_CONFIG_KEY=tfstate-backend/terraform.tfstate
 ✓  (root-admin) tfstate-backend ⨠  terraform init
Initializing modules...
- module.tfstate_backend
- module.tfstate_backend.s3_bucket_label
- module.tfstate_backend.dynamodb_table_label

Initializing the backend...
bucket
  The name of the S3 bucket

  Enter a value:

Erik Osterman (Cloud Posse)over 7 years ago

zoom?

Erik Osterman (Cloud Posse)over 7 years ago

https://zoom.us/j/4155358615

Rohit Vermaover 7 years ago

sure

Tobias Hoellrichover 7 years ago(edited)

Good afternoon #terragrunt - I have an odd situation I can't explain and was wondering if one of you had any insight.

Tobias Hoellrichover 7 years ago

Tobias Hoellrichover 7 years ago(edited)

~~Any ideas why plan-all in the root wants to create resources that already exist?~~ This turned out to be a stupid mistake where I had created a new component in the live directory and it had a copy of the wave terraform.tfvars in it ... 😞

January 2019 Browse by date March 2019