#aws - November 2019

terraform is replacing the instance while enabling ebs encryption after creation of the instance. Any way to avoid this?

Should we use kms_key_id (arn) or the kms_alias as a value for the kms_key_id while encrypting root volume? If I use the alias, terraform is recreating the instance saying the kms key id is different from the last time.

Anyone here runs R5 AWS instance type for kubernetes ? Our fleet runs pretty high on memory and low on CPU, and I wonder if switching to R5 would be a good cost effective move. Is R5 cpu usually good enough to support a kubernetes with multiple web services (with low to mid CPU consumption) ?

Hey team, looking at the autoscale-group module as called from eks-workers module.
I see in Console that launch_template_version default isn't being set to the latest. I have v1 (default) and v2.
$Latest is the default in the autoscale-group module though: https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/blob/master/variables.tf#L78
eks-workers module doesn't supply launch_template_version value to autoscale-group so figure it should use default (of $Latest)
I'm trying to work out how nodes will rotate with an update to launch template and validating blue/green worker pools.
$Latest seems to be correct per terraform provider docs: https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html#with-latest-version-of-launch-template

people using multi account and ecr + eks: how are you distributing images between accounts?
i.e. are you
a) allowing accounts to pull from a central repository
or
b) pushing images to multiple accounts
---
follow up question: what hurts about the option you've gone with?

Good question. On a recent engagement, we went with option (a), but considered both. It was just so much easier to have a centralized repo rather than worry about promotion.

@Igor Rodionov can answer technical questions.

how painful was management of cross account repo policies?

are you doing multiregion ?

single region

so multi aws account one ECR ?

yeah - that's option a

basically ecr hosted within a shared 'infra' account and different clusters pull from that

option b is that images are pushed to different accounts based on 👋 processes 👋

@Chris Fowles we use A

We are in multi region but we use the “regional account ecr” as a central repo

so same as having option A

yeh cool

cheers

it is such a pain to push images to many repos

I was leaning towards A for that reason

thanks folks 🙂

Hello there, how are you? Could you please tell me if u have any experience using cloudwatch anomaly detection? I couldn't find this resource in terraform docs.

I kept my images in gitlab 😅

that is running inside one my ECS instances in AWS

I did not have to do anything access wise, it is done trough a tunnel to github

no user input needed just the command to run the installer ( it is in beta)

but the fact I can run this means I could start a container with the software needed, spin it up upon action in git wait until the hosted runner service is up and then run terraform apply that will be using the credentials given by IAM profile to specific resources, limiting what the runner can do.
This is a pretty cool new feature of githug actions self hosted runners

Awesom

@PePe Amengual I guess it can be run in Fargate as well ?

as long as you run the executable ( compiled from source) then yes

as the entrypoint

my guess is they will have a docker image soon

so you need GH actions to compile GH actions 🙂

lol

Question : is Cloudposse going to be at re:Invent ?

Anybody knows a tool, shell or GitHub (python) project which helps me working within AWS organizations to rollout a bunch of settings across all accounts and regions? Especially for those services which aren't integrated into AWS orgs. I'd like to use assume-role into the target account.

Have you looked at organisation group policies? SCP

SCPs don't configure services.

well, only a few

Anyone do this yet? https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL-certificate-rotation.html

If anybody has recommendations as to how to best implement DLQs w/Lambdas I'd appreciate input -- trying to figure out if I should have a single target by Lambda, by region, by account or have a single target for my org. I'm thinking that I want to "default" to a SNS topic by region.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-supported-resources.html finally!!!!

Amazing! Thanks for sharing. This is a must have.

yeah, would be a cool project around the idea, tracking/matching the event patterns and managing the cloudwatch event rule

Would also like to see more meta data

In the slack alert

Looking for a solution to automatically tag AWS resources that were created with the creator and time . I came across this project https://github.com/GorillaStack/auto-tag .

Anyone else have recommendations?

This can be used for instances only
http://answersforaws.com/code/graffiti-monkey/

That looks like it only transfers existing tags from the instance to volumes/snapshots?

Hem, does AWS network load balancing works between instances? I have insance A (part of asgA) and instance B(part of asgB). I need to connect from A to non http endpoint in B. I am trying to do it with NLB but it does not work. I am starting to wonder if i am missing something - maybe it cannot be done that way? (Security groups on both A and B allows whole 10/8 network and all servers are in it.)

This looks really strange - when i try to telnet from A to B it stays around 3-5 minutes in Trying.. and then it connects. Till that time i can see only Syn packages being sent from source host and no packages on target host. Once connection is established i can see first packages reaching target

has anyone been able to terraform out inspector in an automated fashion?

inspector ?

aws inspector

im having issues finding good examples

I have not used, just guard duty and config

I have those both working well, was wondering about inspector. its cool

Hi

I’m looking for a solution to monitor changes in AWS

Anyone knows a solution? preferable open source one

AWS Config ?

Terraform plan?

Config would be your best bet - it will inform you of changes from your BASELINE report created at the start, any changes from that such as s3 bucket permissions changed it will notify you so that you can resolve.

Hi all, first time poster here so be gentle 😄,

I have a question about a problem we have regarding AWS accounts, IAM policies, and S3 permissions in general.

So, as an example:
• we have a single AWS account
• multiple EKS clusters, one per environment, under the same AWS account
• each EKS cluster has a role attached for the worker nodes
• a kubernetes workload will upload files from any public S3 bucket and save it in to the clusters own private S3 bucket

The challenge:
• the workload should be able to read from the public buckets from another environment under the same account.
• the workload should not be able to read from the private buckets from another environment under the same account.

Is there an IAM policy to achieve this? I have looked at https://docs.aws.amazon.com/AmazonS3/latest/dev/walkthrough1.html but this would entail explicitly Denying access to other specific buckets.

Or are all buckets, by the nature of things, always visible under a single AWS account with the s3:ListAllMyBuckets permission.

Is there another way to achieve this? Are there any best practices? Using an account per environment would enable us to restrict by account number.

https://github.com/aws/containers-roadmap/issues/583

The one we talked yesterday

Eks node groups with SPOT

Thanks!!

Registry + Resource Providers + Third-Party support... starting to be more competitive with Terraform 😄

Hey all - I am looking for a way to debug and develop an api gateway lambda - I’m using Sam local with mock data, but I feel there is a better way - is anybody using something different? Any way to capture the request data sent to api gateway for easier access?

Hey guys,

I am trying to create an AWS config custom rule which checks that a certain setting is always on (EBS default encryption)

When submitting my evaluation using the put_evaluations API call, I am required to submit a resourcetype and resourceid

Now, I’m monitoring a setting and not a resource… I’m at a loss as to what resource I ought to specify here. Anyone able to help?

Hello there! I'm trying to run aws-nuke solution, I could delete one account but I had some issues trying to connect with another account using cross account role. somebody have any experience with this tool? Maybe a config file example?

Hi, I'm planning on setting up Vault for our credential management. Anyone here with experience doing that? Would love to hear if you like it, common pitfalls you ran into or other things I might have to take into consideration for this project.

i both loath and love the lead-up to re-invent

I’ve never been, I’m kinda excited.

i'm not heading there this year - i'm more talking about the amount of stuff that changes in aws in a very short time that you need to get your head around

it was worse when i was consulting - at least now i don't have to worry about being ambushed by questions about a 2 hour old service when walking into a pre-sales meeting 😂

Hahaha oh man yeah

I am looking on some guidance using secrets with EKS. I could use K8s secrets and mount them on a volume (EFS ? ). I would prefer use of Parameter Store/Secrets Manager.

i've not found anything perfect for my use case in deal with secrets pulled from paramstore so i'm using a small bootstrapper that i wrote to pull secrets from paramstore and inject them into env vars in process

similar to a chamber exec https://github.com/segmentio/chamber#exec

didn't need all of chamber though so wrote our own

it feels as though this should be a solved problem, but i wasn't able to find a tool that did exactly what i needed

if you're ok with secrets being written to etcd then you can use something like https://github.com/godaddy/kubernetes-external-secrets

Thank you for sharing ..I'll take a look .. I do see this as an item on the EKS roadmap, it's still being researched though

quick question, can I add a new tag using terraform module with the volume-id I get from Data source ?