#geodesic - August 2019 | Slack Archive

123 messages

Discussions related to https://github.com/cloudposse/geodesic

Archive: https://archive.sweetops.com/geodesic/

Jul 2019

August 2019

Sep 2019

Tim Jonesabout 7 years ago

Ah, I think I see it now, create the individual users with terraform-aws-iam-user, manage the settings of all the users with terraform-aws-iam-assumed-roles and delegate their account access with terraform-aws-organization-access-group`

Erik Osterman (Cloud Posse)about 7 years ago

the terraform-aws-organization-access-group is somewhat redundant with terraform-aws-iam-assumed-roles

Erik Osterman (Cloud Posse)about 7 years ago

the terraform-aws-organization-access-group module was designed to be a more general way to assume any role in other accounts

Erik Osterman (Cloud Posse)about 7 years ago

while the terraform-aws-iam-assumed-roles is designed specifically to assume roles in the "root" (apex) account.

Tega McKinneyabout 7 years ago(edited)

Is it possible override the terraform backend s3 key? I created the following top-level folder structure:


root
-- vpc
-- -- terraform.envrc

I would like the key to be : root/vpc/terraform.tfstate so I set TF_CLI_INIT_BACKEND_CONFIG_KEY=root/vpc/terraform.tfstate.

This does not override the default cause with the key ends up being vpc/terraform.tfstate. I sit possible to override the key?

Tega McKinneyabout 7 years ago

Ran into another issue where I have created additional root-dns entries that have a hyphen in the name. The label is removing the hyphen from the stage however I assumed the regex_replace_chars should not replace hyphens. Is there something else that could be stripping the hyphen?

Tega McKinneyabout 7 years ago

Regarding helmfiles, is multi-stage dockerfiles the current approach? How does that relate to /templates/conf/helmfiles/... in reference-architectures?

Erik Osterman (Cloud Posse)about 7 years ago

great you ask

Erik Osterman (Cloud Posse)about 7 years ago

no - we're using remote helmfiles pinned to releases now

Erik Osterman (Cloud Posse)about 7 years ago

It looks like this:

Erik Osterman (Cloud Posse)about 7 years ago


# Ordered list of releases.
# Terraform-module-like URLs for importing a remote directory and use a file in it as a nested-state file.
# The nested-state file is locally checked-out along with the remote directory containing it.
# Therefore all the local paths in the file are resolved relative to the file.

helmfiles:
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/reloader.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/cert-manager.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/prometheus-operator.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/cluster-autoscaler.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/kiam.yaml?ref=0.51.2>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/external-dns.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/teleport-ent-auth.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/teleport-ent-proxy.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/aws-alb-ingress-controller.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/kube-lego.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/nginx-ingress.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/heapster.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/dashboard.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/forecastle.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/keycloak-gatekeeper.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/fluentd-elasticsearch-aws.yaml?ref=0.47.0>
  - path: git::<https://github.com/cloudposse/helmfiles.git@releases/kubecost.yaml?ref=0.50.0>

Tega McKinneyabout 7 years ago

Okay; reason I was asking is I attempted to add /helmfiles/ templates to the configs/prod.tf in reference-architecture. Failed to render templates as it was looking for a few env vars that are not setup (KOPS_CLUSTER_NAME and STAGE). That got me thinking those templates are just reference files for after the initial architecture is setup. Is that the general idea?

Erik Osterman (Cloud Posse)about 7 years ago

So the Helmfiles will require a ton of settings

Erik Osterman (Cloud Posse)about 7 years ago

We do not have those documented. But if you share specific ones I can help.

Erik Osterman (Cloud Posse)about 7 years ago

Usually we set STAGE in the Dockerfile. If you only have one cluster in a stage, then you can also set KOPS_CLUSTER_NAME in the Dockerfile so it's available globally.

SweetOps #geodesicabout 7 years ago

There are no events this week

Cloud Posseabout 7 years ago

Join us for "Office Hours" every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about INLINECODE_0, get live demos and learn from others using it. Next one is Aug 14, 2019 11:30AM.

👉️ Register for Webinar

#office-hours (our channel)

Tim Jonesabout 7 years ago

Hi! I've been reviewing the terraform-root-modules resource in order to help bring a little order to the chaos AWS I've inherited but I'm having trouble understanding where real human users are managed.
I see that the aws/usersseems to be set up for this, but it seems incomplete or something, the welcome.txt references a username var that doesn't exist, and isn't used as a template source as far as I can see anyways.

Dave Yuabout 7 years ago

I’m having trouble building from geodesic with terraform 0.11 since 0.117.0

Dave Yuabout 7 years ago

is there a better way to use both 0.11 and 0.12?

Erik Osterman (Cloud Posse)about 7 years ago

Yes, but I am afk

Erik Osterman (Cloud Posse)about 7 years ago

@Andriy Knysh (Cloud Posse) can you show Dave

Erik Osterman (Cloud Posse)about 7 years ago

Basically install terraform_0.11@cloudposse

Erik Osterman (Cloud Posse)about 7 years ago

And write “use terraform 0.11”

Erik Osterman (Cloud Posse)about 7 years ago

In your .envrc

Dave Yuabout 7 years ago

where/when do I install 0.11? before geodesic 0.117.0, apk add terraform_0.11@cloudposse worked in the Dockerfile

Erik Osterman (Cloud Posse)about 7 years ago

Yep, add that to your Dockerfile

Erik Osterman (Cloud Posse)about 7 years ago

This way we can support multiple concurrent major/minor versions

Dave Yuabout 7 years ago

Yes, but during docker build:

ERROR: unsatisfiable constraints:
  terraform-0.12.0-r0:
    breaks: world[terraform=0.11.14-r0]
The command '/bin/sh -c apk add terraform_0.11@cloudposse terraform@cloudposse==0.11.14-r0' returned a non-zero

Erik Osterman (Cloud Posse)about 7 years ago

Show me your Dockerfile

Erik Osterman (Cloud Posse)about 7 years ago

You need to remove the second package there

Erik Osterman (Cloud Posse)about 7 years ago

RUN apk add terraform_0.11@cloudposse==0.11.14-r0 is what you want

Erik Osterman (Cloud Posse)about 7 years ago

the long & short of it is that since we upgraded to the alpine:3.10 series, theres been no new 0.11 release, so no package for 0.11 was built under terraform.

Erik Osterman (Cloud Posse)about 7 years ago

however, we explicitly build a terraform_0.11 package and a terraform_0.12 package

Erik Osterman (Cloud Posse)about 7 years ago

like a python2 and python3 package

Erik Osterman (Cloud Posse)about 7 years ago

and from alpine:3.10 , the terraform package will be 0.12.x

Erik Osterman (Cloud Posse)about 7 years ago

behind the scenes, we're installing a symlink to /usr/local/terraform/x.y/bin/terraform that points to /usr/local/bin/terraform-x.y

Erik Osterman (Cloud Posse)about 7 years ago

that way when we write use terraform 0.11, we can set PATH=/usr/local/terraform/0.11/bin:$PATH and it will automatically find the correct version of terraform

Erik Osterman (Cloud Posse)about 7 years ago

without changing code

Erik Osterman (Cloud Posse)about 7 years ago

and not using alias

Andriy Knysh (Cloud Posse)about 7 years ago

@Dave Yu this is what we used in Dockerfile to install both TF 0.11 and 0.12 under geodesic 0.117

Andriy Knysh (Cloud Posse)about 7 years ago

# Install terraform 0.11 for backwards compatibility
RUN apk add terraform_0.11@cloudposse

# Install terraform 0.12
RUN apk add terraform_0.12@cloudposse terraform@cloudposse==0.12.3-r0

Andriy Knysh (Cloud Posse)about 7 years ago

then, for the modules that use TF 0.12, we use

use envrc
use terraform 0.12
use tfenv

Andriy Knysh (Cloud Posse)about 7 years ago

and for the modules that use TF 0.11

use envrc
use terraform 0.11
use tfenv

Dave Yuabout 7 years ago

thanks @Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse) RUN apk add terraform_0.11@cloudposse was the trick (instead of RUN apk add terraform_0.11@cloudposse terraform@cloudposse==0.11.14-r0)

Erik Osterman (Cloud Posse)about 7 years ago

#office-hours hours starting in 15m
https://zoom.us/meeting/register/dd2072a53834b30a7c24e00bf0acd2b8

Tega McKinneyabout 7 years ago

Has any configured kiam beyond the helmfiles defaults for cross account resource access from kops pods? I’m running into a situation where my cross account policies are not allowing me access and I’m thinking maybe kiam is not properly setup to assume roles

Erik Osterman (Cloud Posse)about 7 years ago

have not tried to accomplish cross account kiam

Erik Osterman (Cloud Posse)about 7 years ago

generally we try never to cross account boundaries

SweetOps #geodesicabout 7 years ago

There are no events this week

Cloud Posseabout 7 years ago

👉️ Register for Webinar

#office-hours (our channel)

Ryanabout 7 years ago

Hello all, I’m playing with geodesic and it doesn’t seem to be generating the dynamic cluster correctly. I get output that is example.foo.bar.


$ export CLUSTER_NAME=<http://test.myco.com|test.myco.com>

$ docker run -e CLUSTER_NAME \
-e DOCKER_IMAGE=cloudposse> -e DOCKER_IMAGE=cloudposse/${CLUSTER_NAME} \
-e DOCKER_TAG=dev \
cloudposse/geodesic:latest -c new-projec> -e DOCKER_TAG=dev \
> cloudposse/geodesic:latest -c new-project | tar -xv -C .
Building project for example.foo.bar...
./
example.foo.bar/Dockerfile
example.foo.bar/Makefile
example.foo.bar/conf/
example.foo.bar/conf/.gitignore

I’m looking through various source, it’s pretty challenging to piece it all together (feels over abstracted). So wondering if I’m missing something.

Ryanabout 7 years ago

Also new to Terraform so the concepts for how it’s organized is a bit confusing atm

Erik Osterman (Cloud Posse)about 7 years ago

Hey Ryan - sorry - our docs are really out of date.

Erik Osterman (Cloud Posse)about 7 years ago

Here's an example of how we use it: https://github.com/cloudposse/testing.cloudposse.co

SweetOps #geodesicabout 7 years ago

There are no events this week

Cloud Posseabout 7 years ago

👉️ Register for Webinar

#office-hours (our channel)

oscaralmost 7 years ago(edited)

Any thoughts on why I receive the following error:


 ✗ . (none) backend ⨠ terraform init
Copying configuration from "git::<https://github.com/cloudposse/terraform-root-modules.git//aws/tfstate-backend?ref=tags/0.35.1>"...

Error: Can't populate non-empty directory

The target directory . is not empty, so it cannot be initialized with the
-from-module=... option.

Latest Geodesic.

oscaralmost 7 years ago


 ✗ . (none) backend ⨠ ls -la
total 16
drwxr-xr-x 2 root root 4096 Aug 22 15:00 .
drwxr-xr-x 3 root root 4096 Aug 22 09:49 ..
-rw-r--r-- 1 root root  380 Aug 22 15:29 .envrc
-rw-r--r-- 1 root root  122 Aug 22 15:00 Makefile
 ⧉  oscar
 ✗ . (none) backend ⨠ cat .envrc
# Import the remote module
export TF_CLI_INIT_FROM_MODULE="git::<https://github.com/cloudposse/terraform-root-modules.git//aws/tfstate-backend?ref=tags/0.35.1>"
export TF_CLI_PLAN_PARALLELISM=2

source <(tfenv)

use terraform 0.12
use tfenv

Andriy Knysh (Cloud Posse)almost 7 years ago

add export TF_MODULE_CACHE=.module

joshmyersalmost 7 years ago

@oscar It has been discussed here a while back, check through history, as @Andriy Knysh (Cloud Posse) says ^^

joshmyersalmost 7 years ago

hey @Andriy Knysh (Cloud Posse) 👋

oscaralmost 7 years ago

Thanks both, I did actually have a scan through 0.12 and geodesic channel but couldn’t find it.

Andriy Knysh (Cloud Posse)almost 7 years ago

and in Makefile.tasks, change it to:

Andriy Knysh (Cloud Posse)almost 7 years ago

-include ${TF_MODULE_CACHE}/Makefile

deps:
	mkdir -p ${TF_MODULE_CACHE}
	terraform init

## Reset this project
reset:
	rm -rf ${TF_MODULE_CACHE}

oscaralmost 7 years ago

Ah you know I can see it now

oscaralmost 7 years ago

19th July

joshmyersalmost 7 years ago

https://github.com/cloudposse/geodesic/pull/500

oscaralmost 7 years ago

Thanks just had a read of that

oscaralmost 7 years ago

Makes sense. I must have missed this this afternoon!

oscaralmost 7 years ago

is source <(tfenv) required?

oscaralmost 7 years ago

@Andriy Knysh (Cloud Posse) I get this from the new Makefile


 ✗ . (none) backend ⨠ make
Makefile:4: *** missing separator.  Stop.
 ⧉  oscar
 ✗ . (none) backend ⨠ make reset
Makefile:4: *** missing separator.  Stop.
 ⧉  oscar
 ✗ . (none) backend ⨠ make deps
\Makefile:4: *** missing separator.  Stop.

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

Makefile:4: *** missing separator. Stop. is usually causes by spaces (replace with tabs)

oscaralmost 7 years ago

Thanks =] All working now

SweetOps #geodesicalmost 7 years ago

There are no events this week

Cloud Possealmost 7 years ago

👉️ Register for Webinar

#office-hours (our channel)

joshmyersalmost 7 years ago

How are folks doing vanity domains using geodesic / ref arch ?

joshmyersalmost 7 years ago

Currently the parent dns name (foo.com) lives in the root account, and by default, users don’t have admin in root

joshmyersalmost 7 years ago

So no one can actually delegate example.foo.com down to example.prod.foo.com

joshmyersalmost 7 years ago

Easily solvable but wondering what folks are doing

Erik Osterman (Cloud Posse)almost 7 years ago

by design, the branded domain is provisioned in the root account, since it will contain references to the service discovery domain in any account.

Erik Osterman (Cloud Posse)almost 7 years ago

e.g. corp/shared account, prod account, data account, etc.

Erik Osterman (Cloud Posse)almost 7 years ago

branded = vanity

Erik Osterman (Cloud Posse)almost 7 years ago


terraform {
  required_version = "~> 0.12.0"

  backend "s3" {}
}

provider "aws" {
  version = "~> 2.17"

  assume_role {
    role_arn = var.aws_assume_role_arn
  }
}

variable "aws_assume_role_arn" {
  type = string
}

data "aws_route53_zone" "ourcompany_com" {
  # Note: The zone name is the domain name plus a final dot
  name = "ourcompany.com."
}

# allow_overwrite lets us take over managing entries that are already there
# use sparingly in live domains unless you know what's what
resource "aws_route53_record" "apex" {
  allow_overwrite = true
  zone_id         = data.aws_route53_zone.ourcompany_com.zone_id
  type            = "A"
  ttl             = 300
  records         = ["1.2.3.4"]
}

resource "aws_route53_record" "www" {
  allow_overwrite = true
  zone_id         = data.aws_route53_zone.ourcompany_com.zone_id
  type            = "CNAME"
  ttl             = 300
  records         = ["<http://app.prod.ourcompany.org|app.prod.ourcompany.org>"]

joshmyersalmost 7 years ago

Right, but using the ref arch, even admin users in the root account don’t have access to do anything to route53 resources

oscaralmost 7 years ago

Yes similar to above I would create a R53 role in root account and assume the role using a second provider block and then for the r53 record resource use the root account t provider.

joshmyersalmost 7 years ago

Why not add route53 perms to the root admin group that already exists?

joshmyersalmost 7 years ago

Currently the users in root admin group, in the root account can only do IAM stuff for their own users

joshmyersalmost 7 years ago

Ah, nevermind me 🤦

oscaralmost 7 years ago

What was the issue/thing for future reference?

joshmyersalmost 7 years ago

Me being stupid and missing the admin role

Erik Osterman (Cloud Posse)almost 7 years ago

#office-hours starting now! join us here https://zoom.us/s/508587304

oscaralmost 7 years ago

@Erik Osterman (Cloud Posse) what is to stop a team just having one Geodesic module and then having nested .envrc in /conf to simulate global variables? The only disadvantage I spot is the swapping of AWS-crednentials which could be easily solved.

oscaralmost 7 years ago

e.g.
/conf/prod/.envrc (all the account specific vars)

BAU:
/conf/prod/eu-west-1/terraform/my_project/*
/conf/prod/eu-west-2/terraform/my_project/*

oscaralmost 7 years ago

Likewise an interesting conversation came up internally yesterday:

In the same way we would promote code in git dev --> master… how would one promote new variables etc along Geodesic /conf? Or new version pegging in the .envrc from-module line?
The only solution to make it easy is have all the Geodesic modules in one repo so that it would be easily spotted if you missed updating one environment in the PR.

oscaralmost 7 years ago

I feel I have not seen the Cloudposse way

joshmyersalmost 7 years ago

but in geodesic you have all your env vars because you are wrapping geodesic in your env specific container, no?

oscaralmost 7 years ago

Not sure I follow.. but normally you would have acme.stage.com but I wonder why not have:
acme.com

and then:
/conf/
../dev/.envrc (with equivalent variables of Dockerfile)
/prod/.envrc (with equivalent variables of Dockerfile)

oscaralmost 7 years ago

It’s a bit weird but something like that came up in a meeting when I was introducing Geodesic to another team and I didn’t really have an answer other than “that isn’t the way”, but it could work without losing any features so…?

oscaralmost 7 years ago

More of a probing question than ‘how to do this’. Curious to what other’s think, specifically Erik and Jeremy

Alex Siegmanalmost 7 years ago(edited)

The first thing I know would be "broken" is role assumption to AWS. It'd be really easy to assume role in dev, do some stuff, then not be able to apply a different stage because you're still the the dev account.

I think the separation can be healthy even if it's not DRY. It's like the age-old "do we deploy centralized logging per stage or put all stages in one place" kind of argument. There's pros and cons to both, but if you're trying to prevent leakage between environments, why not prevent it at the tool level too

The separation between stages also allows you to test changes to your tool chain in safe environments, rather than break the production one.

joshmyersalmost 7 years ago

Yup, makes CI/CD a bit cleaner too

Alex Siegmanalmost 7 years ago(edited)

I also like that the toolchain follows the same "TRUST THE ENVIRONMENT" we yell to our ~~devs~~ selves for 12-factor style apps, but maybe not everybody follows 12-factor app style.

Erik Osterman (Cloud Posse)almost 7 years ago

on the one hand, I'm jealous of companies that take the monorepo approach to infrastructure. you can easily bring up and tear down your entire environment. You can open PRs that fix problems across all environments in one fell swoop. You can easily setup cross account peering because you control all accounts in a single session. You can do all sorts of automation.

Erik Osterman (Cloud Posse)almost 7 years ago

but this also is freggin scary. i think it's optimizing for the wrong use-case where you start from scratch. i think it's better to optimize for day to day operations and stability.

Erik Osterman (Cloud Posse)almost 7 years ago

also, what i struggle with is where do you draw the line?

Erik Osterman (Cloud Posse)almost 7 years ago

i'm sure most of the engineers would agree "share nothing" is the right approach

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

but despite that, tools, state buckets, repos, dns zones, accounts, webhooks, ci/cd, etc are all shared.

Erik Osterman (Cloud Posse)almost 7 years ago

we've taken the (relatively speaking) extreme position to truly share nothing.

Erik Osterman (Cloud Posse)almost 7 years ago

we don't share the tools, they are in different containers.

Erik Osterman (Cloud Posse)almost 7 years ago

we don't share the state buckets, they are in different accounts.

Erik Osterman (Cloud Posse)almost 7 years ago

we don't share the repos, each account corresponds to a repo

Erik Osterman (Cloud Posse)almost 7 years ago

we don't share DNS zones. each account has it's own service discovery domain

Erik Osterman (Cloud Posse)almost 7 years ago

we don't share webhooks, because each account has it's own repo

Erik Osterman (Cloud Posse)almost 7 years ago

we don't share CI/CD (for infra) because each account has it's own atlantis, and each atlantis receives it's own webhooks

Erik Osterman (Cloud Posse)almost 7 years ago

etc...

Andriy Knysh (Cloud Posse)almost 7 years ago

yes, all of that looks like very difficult to setup and maintain from the start, but in the end it’s much easier to manage security and access without jumping through many hoops (and still having holes)

Andriy Knysh (Cloud Posse)almost 7 years ago

in that share nothing architecture, the only point of access to an account is to add a user to the identity account and allow it to assume a role with permissions to access resources in the other accounts

Andriy Knysh (Cloud Posse)almost 7 years ago(edited)

note that we still share all the code (from terraform modules, helmfiles, and the root-modules catalog) in all account repos, so no repetition there (we load them dynamically using tags with semantic versioning)

Andriy Knysh (Cloud Posse)almost 7 years ago

we just have different settings (vars, ENvs, etc.) for each account

Andriy Knysh (Cloud Posse)almost 7 years ago

as you can see for example here https://github.com/cloudposse/testing.cloudposse.co/tree/master/conf/ecs, there is no code in the account repos, just settings (not secrets)

Andriy Knysh (Cloud Posse)almost 7 years ago

all code (logic) is shared and open (if it’s your company secret, make the repo private)

oscaralmost 7 years ago

Yes that example is similar to how were doing it now.
We have a few levels of abstraction going on. I can go over this next wednesday for 10-15 minutes d

July 2019 Browse by date September 2019