#geodesic - November 2018 | Slack Archive

geodesic

296 messages

Discussions related to https://github.com/cloudposse/geodesic

Archive: https://archive.sweetops.com/geodesic/

Oct 2018

November 2018

Dec 2018

Domboover 7 years ago

What do you guys use to host the packages available at apk.cloudposse.com?

Domboover 7 years ago

Managed service or self hosted?

Erik Osterman (Cloud Posse)over 7 years ago

Good question

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/alpinist

Erik Osterman (Cloud Posse)over 7 years ago

It’s a fork

Erik Osterman (Cloud Posse)over 7 years ago

Very cool little lambda deployed via cloud formation

Erik Osterman (Cloud Posse)over 7 years ago

PR is still open because we’ve been using it for demos

Erik Osterman (Cloud Posse)over 7 years ago

With Atlantis

Domboover 7 years ago

haha dude

Domboover 7 years ago

I was like "I do not want to host an APK repo"

Domboover 7 years ago

THen found this https://github.com/forward3d/alpinist

Domboover 7 years ago

Checked back here to see if you'd hear of it

Domboover 7 years ago

TUrns out you've done the hard part for me already

Erik Osterman (Cloud Posse)over 7 years ago

If you’re going to host one I can finish up the PR next week

Domboover 7 years ago

I'm happy to jump on with you?

Erik Osterman (Cloud Posse)over 7 years ago

I think it’s ready to merge and release

Erik Osterman (Cloud Posse)over 7 years ago

Then our CloudPosse packages repo is how we generate our packages

Domboover 7 years ago

Yeah noted

Erik Osterman (Cloud Posse)over 7 years ago(edited)

We are going to consolidate the vendor and install folders in packages. Our justification for doing it this way is so we don’t have lock in on apk. We use it to install native binaries (e.g. for local Dev) and maybe one day will support other packages like Debian.

Domboover 7 years ago

I might be missing something but I'm unsure how consolidating those folders will change anything? That Makefile in install is a manifest and quite useful as a new-comer to the repo.

Domboover 7 years ago

You can achieve multi-package support by simply adding more steps to the individual Makefiles in the vendor dir

Domboover 7 years ago

Disclaimer: not exactly a Makefile pro but it's easy enough to follow

Erik Osterman (Cloud Posse)over 7 years ago

Oh the install folder was just how we used to do it

Erik Osterman (Cloud Posse)over 7 years ago

The vendor folder is the new way

Erik Osterman (Cloud Posse)over 7 years ago

We are going to add a make target to preserve the original behavior

Erik Osterman (Cloud Posse)over 7 years ago

But keep all the logic in the vendor folder.

Erik Osterman (Cloud Posse)over 7 years ago

Oh btw, if there are any packages missing you would like feel free to open an issue or PR (preferred) and we can add them

Domboover 7 years ago

Yeah was just thinking that - most of my daily tools are there

tamskyover 7 years ago

Does anyone have an example repo that uses the "multi-stage docker build pattern" against geodesic?
I'm hoping not to reinvent the wheel for customizations.

Andriy Knysh (Cloud Posse)over 7 years ago

https://github.com/cloudposse/prod.cloudposse.co

Andriy Knysh (Cloud Posse)over 7 years ago

https://github.com/cloudposse/staging.cloudposse.co

Andriy Knysh (Cloud Posse)over 7 years ago

https://github.com/cloudposse/testing.cloudposse.co

tamskyover 7 years ago

oh, I didn't know those draw from geodesic -- but I should have thought of that

tamskyover 7 years ago

tnx!

Erik Osterman (Cloud Posse)over 7 years ago

😃

tamskyover 7 years ago

what do folks do when they need a later version of a package than what is listed in the cloudposse/packages repo?
I'm looking for a later version of terraform than 0.11.8 listed at https://github.com/cloudposse/packages/blob/master/vendor/terraform/VERSION

tamskyover 7 years ago

or a later version than what's available in the apk aports: https://git.alpinelinux.org/cgit/aports/tree/community/terraform/APKBUILD

Erik Osterman (Cloud Posse)over 7 years ago

Submit a PR against our packages repo

Erik Osterman (Cloud Posse)over 7 years ago

The only reason it’s not the latest is we haven’t had a chance to update it yet.

Erik Osterman (Cloud Posse)over 7 years ago

@tamsky ☝️

Erik Osterman (Cloud Posse)over 7 years ago

Also, anything you want a package for, feel free to open an issue or submit a PR

tamskyover 7 years ago

There seem to be at least two different version directives for terraform within cloudposse/packages:


diff -r 5c58433395f3 install/Makefile
--- a/install/Makefile  Tue Oct 23 18:32:59 2018 -0700
+++ b/install/Makefile  Sun Nov 11 15:53:45 2018 -0800
@@ -297,7 +297,7 @@
 teleport:
        $(CURL) <https://get.gravitational.com/teleport/$(TELEPORT_VERSION)/teleport-v$(TELEPORT_VERSION)-${OS}-$(ARCH)-bin.tar.gz> -o - | tar -C $(INSTALL_PATH) -zx --wildcards --strip-components=1 --overwrite --mode='+x'  */tsh */tctl */teleport
 
-export TERRAFORM_VERSION ?= 0.11.7
+export TERRAFORM_VERSION ?= 0.11.10
 # Releases: <https://github.com/hashicorp/terraform/releases>
 ## Install Terraform
 terraform:
diff -r 5c58433395f3 vendor/terraform/VERSION
--- a/vendor/terraform/VERSION  Tue Oct 23 18:32:59 2018 -0700
+++ b/vendor/terraform/VERSION  Sun Nov 11 15:53:45 2018 -0800
@@ -1,1 +1,1 @@
-0.11.8
+0.11.10

tamskyover 7 years ago(edited)

and there seems to be some conflict between what version of terraform gets installed in geodesic.
geodesic appears to use the apk manifest in //packages.txt to install /usr/bin/terraform.
in the cloudposse/packages image, it installs /usr/local/bin/terraform binary directly from vendor (no apk), but this binary does not get copied into geodesic during the multi-stage build.

tamskyover 7 years ago(edited)

So that may mean there are 3 different version directives of terraform in play?
- //packages/install/Makefile
- //packages/vendor/terraform/VERSION (appears to have no effect)
- //geodesic/packages.txt (link: https://github.com/cloudposse/geodesic/blob/master/packages.txt#L52)

Erik Osterman (Cloud Posse)over 7 years ago

The install folder is preserved for legacy reasons

Erik Osterman (Cloud Posse)over 7 years ago

It will be migrated to use the new make system for backwards compatibility

Erik Osterman (Cloud Posse)over 7 years ago

But for now we just have both

Erik Osterman (Cloud Posse)over 7 years ago

@tamsky i see what you’re saying

Erik Osterman (Cloud Posse)over 7 years ago

//packages/vendor/terraform/VERSION is used by the new system

Erik Osterman (Cloud Posse)over 7 years ago

//packages/install/Makefile we’re going to create a wildcard target to map the old targets to the new targets for backwards compatibility

Erik Osterman (Cloud Posse)over 7 years ago

geodesic appears to use the apk manifest in //packages.txt to install /usr/bin/terraform.

in the cloudposse/packages image,

Erik Osterman (Cloud Posse)over 7 years ago

correct - alpine packages tend to get installed in /usr/bin, so we preserved that

Erik Osterman (Cloud Posse)over 7 years ago

in the cloudposse/packages image, it installs /usr/local/bin/terraform binary directly from vendor (no apk)

Erik Osterman (Cloud Posse)over 7 years ago

this is determined by the INSTALL_PATH environment variable that defaults to /usr/local/bin

Erik Osterman (Cloud Posse)over 7 years ago

we didn’t want non-package packages going into /usr/bin

Erik Osterman (Cloud Posse)over 7 years ago(edited)

/usr/local/bin should have priority over /usr/bin (but i haven’t verified)

Erik Osterman (Cloud Posse)over 7 years ago

i’m open to discussion/suggestions on better ways of handling this

Erik Osterman (Cloud Posse)over 7 years ago

@yurchenko did you get to the bottom of it?

yurchenkoover 7 years ago

@yurchenko has joined the channel

tamskyover 7 years ago(edited)

/usr/local/bin should have priority over /usr/bin (but i haven’t verified)

geodesic's Dockerfile only lists two COPY directives related to the packages image


COPY --from=packages /packages/install/ /packages/install/
COPY --from=packages /dist/ /usr/local/bin/

so everything in --from=packages /packages/bin/ is lost.

tamskyover 7 years ago(edited)

for now, I've fixed my problem of getting v0.11.10 in geodesic by adding
+COPY --from=packages /packages/bin/terraform /usr/bin/terraform

Erik Osterman (Cloud Posse)over 7 years ago

@tamsky see make dist

Erik Osterman (Cloud Posse)over 7 years ago

the idea is that you copy everything you want into /dist

Erik Osterman (Cloud Posse)over 7 years ago

then COPY --from=packages /dist /usr/local/bin

Erik Osterman (Cloud Posse)over 7 years ago

by staging things into the /dist folder, allows you to specify the packages in an ENV, so that you only specify a single COPY statement

Erik Osterman (Cloud Posse)over 7 years ago

if it would help, i can do a quick zoom

Erik Osterman (Cloud Posse)over 7 years ago

@tamsky are you unblocked?

tamskyover 7 years ago

I made good progress yesterday, so I feel pretty good about it all.

Thanks for the reply and offer to help.

tamskyover 7 years ago

allows you to specify the packages in an ENV,

this sounds like a key concept.
I'll take a look at make dist today.

tamskyover 7 years ago

Do you have any existing techniques/strategies for caching the .terraform/plugins/linux_amd64/ directory?
-- maybe as a runtime volume-mount?

Erik Osterman (Cloud Posse)over 7 years ago

i want to do that

Erik Osterman (Cloud Posse)over 7 years ago

i tried it in vane at some point, but the problem is that it’s per terraform project

Erik Osterman (Cloud Posse)over 7 years ago

i wish .terraform/plugins could be mapped to /var/lib/terraform or something

Erik Osterman (Cloud Posse)over 7 years ago

(and without symlinks)

Erik Osterman (Cloud Posse)over 7 years ago

but i haven’t found any envs to do that

tamskyover 7 years ago

i tried it in vane at some point, but the problem is that it’s per terraform project

it could be a greedy/opportunistic algorithm... all local images that match a pattern get mounted somewhere and an init job/script links/copies them into $HOME/.terraform/plugins...

(and without symlinks)

sounds like you know something here. does terraform require the inodes in .terraform/plugins/linux_amd64/ to be regular files and not symlinks?

Erik Osterman (Cloud Posse)over 7 years ago(edited)

no, i’m just “anti-symlink” hacks

Erik Osterman (Cloud Posse)over 7 years ago

especially where generating symlinks programmatically

Erik Osterman (Cloud Posse)over 7 years ago

if it’s a one shot deal, like ln -s /lib64 /lib, thats cool

Erik Osterman (Cloud Posse)over 7 years ago

i guess it’s more related to tfstate. we used to link tfstate to a persistent volume

Erik Osterman (Cloud Posse)over 7 years ago

but if the symlinks were stale or acidentally pointing to the wrong place, it can have catastrophic outcomes. .terraform/plugins should be pretty safe to symlink. Just wish there was an ENV for it.

Erik Osterman (Cloud Posse)over 7 years ago

the init-terraform script would be the place to do it.

tamskyover 7 years ago

well, tbh, I'd expect any mistakes in the tree or data below .terraform/plugins to be ignored by the terraform binary....
Seems like it's an opportunistic cache, and the cache itself does not need versioning... since each provider binary within the cache is versioned.

joshmyersover 7 years ago

https://github.com/hashicorp/terraform/pull/16000

joshmyersover 7 years ago

Does TF_PLUGIN_CACHE_DIR do what you want?

Erik Osterman (Cloud Posse)over 7 years ago

wha?? i missed that

Erik Osterman (Cloud Posse)over 7 years ago

that’s cool

Erik Osterman (Cloud Posse)over 7 years ago

yea, that’s what i want.

Erik Osterman (Cloud Posse)over 7 years ago

now can we have the same for modules? 😃

tamskyover 7 years ago

only if you trust .terragrunt-cache

tamskyover 7 years ago

oh, maybe we can setup geodesic to bind mount the local user's .terraform/plugins, and that way the local user's cache can be shared?

Erik Osterman (Cloud Posse)over 7 years ago

just set TF_PLUGIN_CACHE_DIR=/localhost/.terraform/plugins

Erik Osterman (Cloud Posse)over 7 years ago

we always mount $HOME to /localhost

Erik Osterman (Cloud Posse)over 7 years ago

(so we can use .ssh among other things)

Erik Osterman (Cloud Posse)over 7 years ago

i caution though against using /localhost/.terraform, unless your host os is linux

Erik Osterman (Cloud Posse)over 7 years ago

since you’ll have conflicts (possibly) unless plugins are versioned by arch

Erik Osterman (Cloud Posse)over 7 years ago

maybe use /localhost/.geodesic/terraform

tamskyover 7 years ago

plugins are versioned by arch

they are...

Erik Osterman (Cloud Posse)over 7 years ago

in the filenames?

tamskyover 7 years ago

in the tree structure

Erik Osterman (Cloud Posse)over 7 years ago

nice

tamskyover 7 years ago

this works... my mac's homedir .terraform.d/plugins used to only have a darwin_amd64/ dir, now it also has a new linux_amd64/ dir

tamskyover 7 years ago(edited)

now if we can just get rid of the ssh-add passphrase prompt

Erik Osterman (Cloud Posse)over 7 years ago

I think we should add ENV TF_PLUGIN_CACHE_DIR=/localhost/.terraform.d/plugins to the main geodesic

Erik Osterman (Cloud Posse)over 7 years ago

thoughts?

Andriy Knysh (Cloud Posse)over 7 years ago

@OScar continuing this thread https://sweetops.slack.com/archives/CB3579ZM3/p1542300152238500

Andriy Knysh (Cloud Posse)over 7 years ago

what other events that you want to log are you asking for?

Erik Osterman (Cloud Posse)over 7 years ago

🎉🍻💯

Erik Osterman (Cloud Posse)over 7 years ago

as always, curious if you have any more suggestions 😃

tamskyover 7 years ago

I think the reason I haven't had many suggestions is because I haven't been able to use it daily.
That'll change now. 😃

Erik Osterman (Cloud Posse)over 7 years ago

haha, ok, preparing to duck for cover!

lvhover 7 years ago

@lvh has joined the channel

Erik Osterman (Cloud Posse)over 7 years ago

@tamsky I've updated https://github.com/cloudposse/packages

Erik Osterman (Cloud Posse)over 7 years ago

so that now install/ targets are just proxied to the vendor/<package>/install

Erik Osterman (Cloud Posse)over 7 years ago

(all backward compatible)

joshmyersover 7 years ago

What are folks thoughts on Terragrunt usage in Geodesic? I note some usage in places but not everywhere…

Erik Osterman (Cloud Posse)over 7 years ago

We haven’t committed to it wholeheartedly. Generally averse to wrappers as they often break interoperability.

Erik Osterman (Cloud Posse)over 7 years ago

But I do like how we can extend root modules (e.g. for adding users)

Erik Osterman (Cloud Posse)over 7 years ago

It could rather trivially be used everywhere in geodesic

Erik Osterman (Cloud Posse)over 7 years ago

To date, we have been Docker multi stage to copy and keep things dry

Erik Osterman (Cloud Posse)over 7 years ago

I know one company moved to git submodules

Erik Osterman (Cloud Posse)over 7 years ago

So I think we have 3 good approaches with various trade offs

Erik Osterman (Cloud Posse)over 7 years ago

But it’s a topic that I like to debate. I don’t feel like it’s 100% solved.

Erik Osterman (Cloud Posse)over 7 years ago

Part of what we want is a solution that is tool agnostic, which is why terragrunt is not ideal

Erik Osterman (Cloud Posse)over 7 years ago

We need to version kops, Helmfiles, and other things. Terraform isn’t the end-all-be-all

joshmyersover 7 years ago

Indeed. adding more layers to geodesic such as Terragrunt make things harder to rationalise

joshmyersover 7 years ago

Although I totally see why it is helpful >_<

Erik Osterman (Cloud Posse)over 7 years ago

Part of my motivation for adding it was to show how flexible the strategy is that we have

Erik Osterman (Cloud Posse)over 7 years ago

That it works with terragrunt like it works with everything else

joshmyersover 7 years ago

There are aspects of Terragrunt I don’d mind leaning on too much. Others like the overrides, I’m not so sold on.

Erik Osterman (Cloud Posse)over 7 years ago

I like the easy state auto init, env mapping, and ability to create some thing like a poor mans overlay file system (are those the overrides?)

joshmyersover 7 years ago

In terms of the directory structure and recursively looking up a directory until it reaches a terraform.tfvars, allows only 2 level deep child/parent overrides IIRC. Agree on above likes, and before/after hooks

joshmyersover 7 years ago

Feels weird using it when there is some overlap/similar things going on with geodesic itself

Erik Osterman (Cloud Posse)over 7 years ago

Yea so the terragrunt directory structure doesn’t make sense with geodesic reference architectures

Erik Osterman (Cloud Posse)over 7 years ago

We recommend sticking with our poly repo approach

tamskyover 7 years ago

@joshmyers when you say "overrides" are you talking about terraform's *override.tf feature?

tamskyover 7 years ago(edited)

recursively looking up a directory until it reaches a terraform.tfvars, allows only 2 level deep child/parent overrides

I do this, but in a more generic way in the Makefile I use. It just looks for a placeholder filename that indicates the root of the environment:

export UPSTREAM_ROOT_PREFIX := $(shell FILE=.environment_root ; for i in . .. ../.. ../../.. ; do test -e $$i/$$FILE && echo "$$i/" && exit 0 ; done ; echo "unable-to-find-$$FILE" ; exit 1 )

- https://github.com/tamsky/terrabase/blob/master/root-account-email%40example.com%3D123456789012/prod/Makefile#L3

joshmyersover 7 years ago

https://github.com/gruntwork-io/terragrunt/blob/master/README.md#find_in_parent_folders

Erik Osterman (Cloud Posse)over 7 years ago

We do something similar with the build-harness "autodiscovery"

Erik Osterman (Cloud Posse)over 7 years ago


export BUILD_HARNESS_PATH ?= $(shell until [ -d "$(BUILD_HARNESS_PROJECT)" ] || [ "`pwd`" == '/' ]; do cd ..; done; pwd)/$(BUILD_HARNESS_PROJECT)

Erik Osterman (Cloud Posse)over 7 years ago(edited)

nice to see someone love make as much as we do 😃

Erik Osterman (Cloud Posse)over 7 years ago


<mailto:root-account-email@example.com|root-account-email@example.com>=123456789012/

account-specific resources are placed within an AWS account directory.
directory names the root account email and the 12-digit AWS account number.

that's kind'a nice

Erik Osterman (Cloud Posse)over 7 years ago

https://sweetops.slack.com/archives/CB3579ZM3/p1542762118320000

Erik Osterman (Cloud Posse)over 7 years ago

so there's often a two-phased approach here since a "virgin" account has no IAM users

Erik Osterman (Cloud Posse)over 7 years ago

so step 1, you use the master AWS account credentials to setup the scaffolding

Erik Osterman (Cloud Posse)over 7 years ago

part of that scaffolding is to provision an IAM user for yourself (and anyone else on the team)

Erik Osterman (Cloud Posse)over 7 years ago

step 2, is you use your personal IAM account to provision everything else

Erik Osterman (Cloud Posse)over 7 years ago

this is the ugly "coldstart" problem...

OScarover 7 years ago

Ok so here is my scenario, I am thinking of doing this in the master account (without using the actual master account):

Login to the root account with the root credentials and do the following:
Create new IAM group cloud_admin
Assign AdministratorAccess policy to the group
Create an IAM user with the name aws_admin
Add the user to the group
Enable MFA for the user (we recommend using Google Authenticator as Virtual MFA device)
Generate Access Key ID and Secret Access Key for the user (we’ll need them to run scripts)

Erik Osterman (Cloud Posse)over 7 years ago

yea, that should be fine

OScarover 7 years ago

So in this scenario for root, anyone in the cloud_admin group, including say my individual account can run the Geodesic stuff right?

OScarover 7 years ago

assuming my individual account has keys and enabled with MFA of course

Erik Osterman (Cloud Posse)over 7 years ago

yep, that should work

OScarover 7 years ago

Thanks @Erik Osterman (Cloud Posse)!!

OScarover 7 years ago

this clarification helps me tremendously in explaining what account is needed to run this stuff

Erik Osterman (Cloud Posse)over 7 years ago

np.. yea, in the end, you just need some kind of "administrator" access. then you can go about provisioning all the other stuff.

OScarover 7 years ago

It looks like this error is only happening when I am inside the Geodesic shell…

Erik Osterman (Cloud Posse)over 7 years ago

I think maybe your AWS_DEFAULT_PROFILE is not set to use that profile?

Andriy Knysh (Cloud Posse)over 7 years ago

Or try to remove the default profile from aws/config, if you have one

OScarover 7 years ago

I don’t have a default profile in ~/.aws/config 😞

Erik Osterman (Cloud Posse)over 7 years ago

Ok one other idea

Erik Osterman (Cloud Posse)over 7 years ago

Run “hwclock -s” in your container

Erik Osterman (Cloud Posse)over 7 years ago

It doesn’t seem to like your MFA token

Erik Osterman (Cloud Posse)over 7 years ago

So maybe clock is out of sync

OScarover 7 years ago

oh you know what? i think you are right

Erik Osterman (Cloud Posse)over 7 years ago

We have seen that happen a lot in Docker for MAC

OScarover 7 years ago

one thing about this account is that the guy who set this up, also setup the mfa so i had to use his barcode in order to add this to my mobile

OScarover 7 years ago

hwclock yields this 2018-11-22 01:41:54.996656+00:00

OScarover 7 years ago

so that time seems to be different than my time

OScarover 7 years ago

its tomorrow

OScarover 7 years ago

for one thing

Erik Osterman (Cloud Posse)over 7 years ago

Though it’s GMT :)

Erik Osterman (Cloud Posse)over 7 years ago

Are the minutes and seconds similar?

OScarover 7 years ago

`✗ (none) ~ ⨠ hwclock -v
hwclock from util-linux 2.32
System Time: 1542851097.345229
Trying to open: /dev/rtc0
Using the rtc interface to the clock.
Last drift adjustment done at 1542851026 seconds after 1969
Last calibration done at 1542851026 seconds after 1969
Hardware clock is on UTC time
Assuming hardware clock is kept in UTC time.
Waiting for clock tick...
...got clock tick
Time read from Hardware Clock: 2018/11/22 01:44:58
Hw clock time : 2018/11/22 01:44:58 = 1542851098 seconds since 1969
Time since last adjustment is 72 seconds
Calculated Hardware Clock drift is 0.000000 seconds
2018-11-22 01:44:56.995687+00:00`

OScarover 7 years ago

but that does not seem like my local time

Erik Osterman (Cloud Posse)over 7 years ago

Did you try hwclock -s?

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/docs/issues/203

OScarover 7 years ago

i did, same issue, the date shows Nov 22 as well

Erik Osterman (Cloud Posse)over 7 years ago

Ok, can help dig in more when in front of a keyboard :)

OScarover 7 years ago

i need a drink!

OScarover 7 years ago

thanks @Erik Osterman (Cloud Posse)!

OScarover 7 years ago

i’ll keep trying things

Erik Osterman (Cloud Posse)over 7 years ago

It’s officially 🍺 o’clock in PST

Erik Osterman (Cloud Posse)over 7 years ago

Maybe some tips in this issue

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/docker/for-mac/issues/1260

OScarover 7 years ago

i found this https://github.com/arunvelsriram/docker-time-sync-agent/ worth a try?

Erik Osterman (Cloud Posse)over 7 years ago

Can try it out...

Erik Osterman (Cloud Posse)over 7 years ago

@tamsky this is adjacent to what you were working on https://github.com/hashicorp/terraform/blob/master/tools/terraform-bundle/README.md

OScarover 7 years ago

@Erik Osterman (Cloud Posse) i tried that agent, no bueno

OScarover 7 years ago

I added this to the Dockerfile and now the date shows correctly, but it is like 2 seconds off!

Janover 7 years ago

Ola!

Janover 7 years ago

quick question, is there a known cost to runt he smallest setup of the full reference archetecture

Janover 7 years ago

with aws org

Andriy Knysh (Cloud Posse)over 7 years ago

@Jan it depends on the project. There is no cost to provision the resources that are just ’metadata` like VPC, subnets, all IAM stuff (users, roles, groups), accounts, etc.

Andriy Knysh (Cloud Posse)over 7 years ago(edited)

for Route53, you just pay for each DNS zone you have (I believe it’s $2 per zone per month)

Andriy Knysh (Cloud Posse)over 7 years ago

then you deploy EC2, RDS, Aurora, Elasticsearch, Elacticache, for which you pay depending on the instance type and count

Andriy Knysh (Cloud Posse)over 7 years ago

if you provision k8s using kops or EKS, you pay per instance type and count

Andriy Knysh (Cloud Posse)over 7 years ago

then, you pay for S3 storage

Andriy Knysh (Cloud Posse)over 7 years ago

and for network traffic

Andriy Knysh (Cloud Posse)over 7 years ago

so, if you deploy the initial setup - VPC, subnets, accounts, IAM, Route53, S3 (for TF and kops states) - you pay just for Route53 and S3 storage (which is almost nothing)

Janover 7 years ago

yea thanks @Andriy Knysh (Cloud Posse)

Janover 7 years ago

im doing an evaluation of full org using geodesic

Janover 7 years ago

and figured I would do so on my own account to start with

Janover 7 years ago

mmm there is something I am not groc'ing

Janover 7 years ago

with aws-vault

Janover 7 years ago

following here https://docs.cloudposse.com/tools/aws-vault/

Erik Osterman (Cloud Posse)over 7 years ago

What’s the error?

Janover 7 years ago

ah nm

Janover 7 years ago(edited)

im being stupid

Janover 7 years ago


aws-vault login aws.tf-staging-admin
Enter passphrase to unlock /Users/xxxxx/.awsvault/keys/:
Enter token for arn:aws:iam::xxxxxxxxxxx:mfa/root-account-mfa-device: xxxx
aws-vault: error: Failed to get credentials for aws.tf-root (source profile for aws.tf-staging-admin): AccessDenied: Roles may not be assumed by root accounts.

Janover 7 years ago

I had the staging account set to the root account id

Janover 7 years ago

😛

Erik Osterman (Cloud Posse)over 7 years ago(edited)

Ah yes copy pasta get me every time

Janover 7 years ago

yarp

Janover 7 years ago

so I would almost do this with terraform local-exec and render templates

Janover 7 years ago

and pass the credentials for boot strap via a .tfvars file which the local exec kills after it runs

Janover 7 years ago

🙂

Janover 7 years ago

also what the docs dont mention is that you should have 2 accounts setup to start with

Janover 7 years ago

😛

Andriy Knysh (Cloud Posse)over 7 years ago

@Jan are you able to login to the root account from geodesic?

Janover 7 years ago

So my current place is:
1. root account is there
2. aws-vault is setup with root admin user
3. I can login or exec
4. its when I then want to do the IAM make init I get promoted for the unset account id var

Janover 7 years ago

I am

Andriy Knysh (Cloud Posse)over 7 years ago

so before #4, let’s provision the other accounts

Janover 7 years ago

Ah I see

Janover 7 years ago

😄

Andriy Knysh (Cloud Posse)over 7 years ago

you have https://github.com/cloudposse/terraform-root-modules/tree/master/aws/accounts

Andriy Knysh (Cloud Posse)over 7 years ago

cd ...

Andriy Knysh (Cloud Posse)over 7 years ago

2. init-terraform

Janover 7 years ago

ok 2 secs

Andriy Knysh (Cloud Posse)over 7 years ago

3. terraform plan <whatever account you need>

Janover 7 years ago

I have 2 dir's

Andriy Knysh (Cloud Posse)over 7 years ago

it will output the account ID

Janover 7 years ago

root and testing

Janover 7 years ago

I am using the root bin now yea?

Andriy Knysh (Cloud Posse)over 7 years ago

then add that account ID to the Dockerfile in the other account’s repo (testing in your case)

Janover 7 years ago

let me quickly do a git reset

Janover 7 years ago

--hard

Janover 7 years ago

I have been making all sorts of changes debugging this

Janover 7 years ago

on a train so internet is slow , gimme a minute

Andriy Knysh (Cloud Posse)over 7 years ago

so in shoer, you go to the root account and provision all other accounts, they will be added to the org automatically since the root account is the master/billing in this case

Janover 7 years ago

that makes sense

Andriy Knysh (Cloud Posse)over 7 years ago

then you update all Dockerfile(s) with the account IDs

Janover 7 years ago

I knew there must be something I was missing

Janover 7 years ago

yea

Andriy Knysh (Cloud Posse)over 7 years ago

then follow the docs https://docs.cloudposse.com/reference-architectures/cold-start/

Andriy Knysh (Cloud Posse)over 7 years ago

sorry for the confusion, we’ll update the docs and ref architectures for that

Janover 7 years ago

alright, let me get back to that point

Janover 7 years ago

hehe all good

Andriy Knysh (Cloud Posse)over 7 years ago

next iteration will be better 🙂

Janover 7 years ago(edited)

I have read most of the docs and loads of the TF / kops / k8s code now

Andriy Knysh (Cloud Posse)over 7 years ago

nice

Andriy Knysh (Cloud Posse)over 7 years ago

if you have any questions or concerns, please just open issues, we’ll get to them ASAP

Janover 7 years ago

btw is there any reason I cant use the same value for parent domain name and namespace?

Janover 7 years ago

Ideally I will send a pull request from fork with the docs stuff

Andriy Knysh (Cloud Posse)over 7 years ago

we use the same namespace for all envs

Janover 7 years ago

some English corrections, missed word etc

Janover 7 years ago

NS yes

Andriy Knysh (Cloud Posse)over 7 years ago

ah yes

Andriy Knysh (Cloud Posse)over 7 years ago

two cases here:

Janover 7 years ago

im using aws.tf as domain and plan to use aws.tf for NS too

Andriy Knysh (Cloud Posse)over 7 years ago

1. Use the same parent domain for all envs, e.g. testing.aws.tf, prod.aws.tf, etc.

Janover 7 years ago

yea totally with you there

Andriy Knysh (Cloud Posse)over 7 years ago

2. Use diff TLS for the envs/accounts, e.g. <http://example.com|example.com> for prod, <http://example.qa|example.qa> for staging etc.

Janover 7 years ago

tld's or tls?

Andriy Knysh (Cloud Posse)over 7 years ago

top-level domains, TLDs

Janover 7 years ago

figured 🙂

Janover 7 years ago

im like a human english linter 😛

Andriy Knysh (Cloud Posse)over 7 years ago

and since those DNS zones are in diff AWS accounts, we use DNS zone delegations - add Name servers for prod.aws.tf to the aws.tf DNS zone

Janover 7 years ago

yea

Janover 7 years ago

with you there too

Janover 7 years ago

so I essentially have done e2e tooling similar to what you have

Janover 7 years ago

over the years many time in TF

Janover 7 years ago

just with less abstraction and less nice docker exec env glue code

Janover 7 years ago

so the entire design idea fits sooooo perfectly with my likes

Andriy Knysh (Cloud Posse)over 7 years ago

yea i see

Janover 7 years ago

really excited to help contribute back

Andriy Knysh (Cloud Posse)over 7 years ago

thanks for testing and using it

Andriy Knysh (Cloud Posse)over 7 years ago

please, any issues or improvements, let us know

Janover 7 years ago

for sure

Janover 7 years ago

have a list I have built up which I will go voer again when there is a new tag for docs

Janover 7 years ago

as it hasnt been deployed since last update

Janover 7 years ago

o.O

Janover 7 years ago


-> Run 'assume-role' to login to AWS
 ⧉  root.aws.tf
 ✗   (none) ~ ⨠  assume-role
Enter passphrase to unlock /conf/.awsvault/keys/:
Enter token for arn:aws:iam::xxxxxxxxxx:mfa/admin: 387674
2018/11/23 14:40:40 Request body type has been overwritten. May cause race conditions
* Assumed role arn:aws:iam::xxxxxxxx:user/admin
* Found SSH agent config
* syslog-ng is already running
* Screen resized to 59x165
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) ~ ⨠

Janover 7 years ago

2018/11/23 14:40:40 Request body type has been overwritten. May cause race conditions

Janover 7 years ago

might be from my train internet

Janover 7 years ago

haha fighting internet here

Janover 7 years ago

🤦

Janover 7 years ago

mmm

Janover 7 years ago

I might just give up and do this when im home in a few hours

Andriy Knysh (Cloud Posse)over 7 years ago

We never tested that on high speed trains :)

Janover 7 years ago

lol

Janover 7 years ago

well I am

Janover 7 years ago

😄

Janover 7 years ago

yea will pick this up later

October 2018 Browse by date December 2018