#aws - April 2019 | Slack Archive

aws

173 messages

Discussion related to Amazon Web Services (AWS)
Archive: https://archive.sweetops.com/aws/

Mar 2019

April 2019

May 2019

rohitover 7 years ago

Is there any advantage in using S3 transfer acceleration if i am already using cloudfront to serve s3 files ?

Maxim Tishchenkoover 7 years ago

hello everyone, is there any way to invoke lambda func, when add/remove user into AWS account ?

Andriy Knysh (Cloud Posse)over 7 years ago

@Maxim Tishchenko you can log user creation/deletion events to CloudTrail https://docs.aws.amazon.com/IAM/latest/UserGuide/cloudtrail-integration.html

Andriy Knysh (Cloud Posse)over 7 years ago

then send all those events to lambda https://docs.aws.amazon.com/lambda/latest/dg/with-cloudtrail.html

Andriy Knysh (Cloud Posse)over 7 years ago

which will filter the required events

Andriy Knysh (Cloud Posse)over 7 years ago

also interesting article https://aws.amazon.com/blogs/mt/automate-account-creation-and-resource-provisioning-using-aws-service-catalog-aws-organizations-and-aws-lambda/

Andriy Knysh (Cloud Posse)over 7 years ago

https://github.com/alphagov/lambda-check-cloudtrail

lvhover 7 years ago

@lvh has joined the channel

Maxim Tishchenkoover 7 years ago

https://aws.amazon.com/premiumsupport/knowledge-center/ebs-charge-stopped-instance/

Maarten van der Hoefover 7 years ago

When an EC2 instance is stopped you only pay for the EBS volumes you are using.

Maarten van der Hoefover 7 years ago

They will when you’re not using elastic ip’s

chrismover 7 years ago

when teleport provide a full script with make file for terraform, but you wish it was just a provider because you only need an 8th of what they've shoved in it https://github.com/gravitational/teleport/tree/master/examples/aws/terraform

chrismover 7 years ago

The UI stuffs nice for teleport but I think it's going to give me a stroke.
We took SSH proxying via a gateway and added 300 new things that can break

chrismover 7 years ago

default size for the SSH proxy in that things a m4.large; 8gig of ram dual-core

chrismover 7 years ago

"sets up the bastion using route53 and registers a cert with letsencrypt" ... so it sticks the route to your bastion into public CT logs (using wildcards is better) and theres no config in that thing to restrict traffic to the bastion sooo we're doing public ssh now

chrismover 7 years ago

all just their example of course; you can do what you like in reality. Wish it was just an ansible module (only one around hasnt been touched in 11 months); at least it doesnt cripple normal ssh

chrismover 7 years ago

so you have a backup when it facepalms

chrismover 7 years ago

@Erik Osterman (Cloud Posse) with the cloudposse bastion what's the deal with users. how can it say which user logged in if the volume mounts against 1 user (just going off the readme)

Erik Osterman (Cloud Posse)over 7 years ago

@chrism user management is outside of the scope of the project. There are dozens of ways to provision users with the #bastion

chrismover 7 years ago

Yeah I mean if i have 9 users already there by other means how does the bastion map to those 9 users if its using volumes?

Erik Osterman (Cloud Posse)over 7 years ago

A bastion is a jump box

chrismover 7 years ago

I know

Erik Osterman (Cloud Posse)over 7 years ago

Users should not be hanging out on it

Erik Osterman (Cloud Posse)over 7 years ago

:-)

chrismover 7 years ago

lol no, but if I create jim, jane, alice with their own auth pubkeys
do I have to map the bastion for all 3 users

Erik Osterman (Cloud Posse)over 7 years ago

See GitHub authorized keys project for inspiration

Erik Osterman (Cloud Posse)over 7 years ago

There’s a gist in the GitHub issues to how someone else did it with cloud formation

chrismover 7 years ago

in comparison; if you setup teleport you add N users; its not mapping 1 auth key; it knows jims keys are x etal

Erik Osterman (Cloud Posse)over 7 years ago

Teleport handles SSO

Erik Osterman (Cloud Posse)over 7 years ago

Teleport is what we use :-)

chrismover 7 years ago

Aye i'm sorta leaning that way at the moment; just not keen on the fat

chrismover 7 years ago

feels like using a juggernaut to deliver a box of matches

Erik Osterman (Cloud Posse)over 7 years ago

We have open sourced our implementation of teleport with kops

Erik Osterman (Cloud Posse)over 7 years ago

Oh yes! It’s totally a hack job to use anything else but teleport

Erik Osterman (Cloud Posse)over 7 years ago

Plus with teleport you get easy YouTube style replays

chrismover 7 years ago

it has lots of selling points; other than having to rejig the world 😛

chrismover 7 years ago(edited)

I assume you can do tedious stuff like create groups with it and say teamx can only access teamx's machines

Erik Osterman (Cloud Posse)over 7 years ago

Yup

Erik Osterman (Cloud Posse)over 7 years ago

You can have groups

Erik Osterman (Cloud Posse)over 7 years ago

Teleport is beautiful. Inside and out.

Erik Osterman (Cloud Posse)over 7 years ago

It is a beast to setup the first time. All in all we have spent probably more than 2 months of man hours on it .

chrismover 7 years ago

Wonder how many headaches getting our "the world is cisco" folks heads around that will be 😄

chrismover 7 years ago(edited)

I just used https://github.com/woohgit/ansible-role-teleport up front; slightly put off for the vsphere land as we use RoyalTS

Erik Osterman (Cloud Posse)over 7 years ago

Are you using dynamodb, IAM roles, s3 backend storage and SAML?

chrismover 7 years ago

teleport would solve lots of the niggly "everything everywhere needs auditing" alongside the "if the grunts dont have ssh access to things I'll have to debug everything"

Erik Osterman (Cloud Posse)over 7 years ago

For teleport auth and node?

chrismover 7 years ago

no; I was using SSH proxy commands in a shell script and a bastion host as the lord-god-aws defined on the mount

chrismover 7 years ago

The ansible script just uses tokens

chrismover 7 years ago

I've setup a proxy/auth on the existing throwaway bastion to test it

chrismover 7 years ago

and shoving a node on another box

chrismover 7 years ago

Their repo's terraform setup for aws consists of graphana/influx monitoring / dynamo/s3

chrismover 7 years ago

our machines are supposed to be immutable; so the only reason you'd ssh in is to grab a log thats not already being exported or to diagnose an issue before burning the machine to the ground

chrismover 7 years ago

We use RKE rather than KOPS

chrismover 7 years ago

which will be more pleasant when ranchers finished its v2 terraform provider that seems to wrap rke + the cli

Erik Osterman (Cloud Posse)over 7 years ago

I am just implying that to do teleport the “right way” will most likely be a lot more work. While getting a POC up takes a day or two. :-)

chrismover 7 years ago

yeah; tbh I just wanted to see how easy it was to throw up on the minimum settings. Nothing comes free 😄

chrismover 7 years ago

the base setup using tokens isn't too bad; realistically as a 1/3 of our setup isn't kubernetes and everything sits in ASGs long-life tokens are probably more necessity than a nicety

chrismover 7 years ago

The rancher hardening guides if they're of interest to anyone https://releases.rancher.com/documents/security/latest/Rancher_Hardening_Guide.pdf

chrismover 7 years ago

sigh; im sold on teleport. Now to read everything 😅

phanindra bollaover 7 years ago

How do i deploy AWS ASG ec2 through terraform as a blue green deployment . i am thinking about diff types of methods
1. Create a Launch template which update/ creates new ASG ,new ALB/ELB and switch the R53 domain to new
2. Create a new Launch template ,, ASG and ALB and update and target ALB to existing R53

please suggest me best way

chrismover 7 years ago

There's numerous things in IAC where you start to think terraform isnt helping 🙄
Create an aws_acm_certificate , then add an extra SAN.
Terraform sits there trying to destroy the old one... but the new ones only just been created; so everythings attached to it, it aint letting go

chrismover 7 years ago

They didnt code in to check the api response that its in use so it just retries to death

chrismover 7 years ago

lmao https://github.com/terraform-providers/terraform-provider-aws/issues/3866 good god its a "feature"

chrismover 7 years ago

Enjoy the weekend chaps

Andriy Knysh (Cloud Posse)over 7 years ago

👍️🍺

keenover 7 years ago

There's numerous things in IAC where you start to think terraform isnt helping

yeah, the more the current crop of tools evolve, the more I miss using my old circa '09 framework that just wrapped the java cli tools......

keenover 7 years ago

"the infra is code because we -wrote- code"

Alex Siegmanover 7 years ago(edited)

I guess the idea is that not everybody has to invent that wheel anymore? I dunno. Certainly understand the feeling though 😃

keenover 7 years ago

yeah - it's just that they write the opinions to be so frameworked...er frameworks to be so opinionated... that if you had a worldview that doesnt fit the framework....

btaiover 7 years ago

thoughts on aurora Postgres? im aware of all the improvements its supposed to give for roughly the same cost over vanilla RDS Postgres but wondering if anyones used it in production and what their thoughts are

Andriy Knysh (Cloud Posse)over 7 years ago

We use it all the time, it’s very good

Andriy Knysh (Cloud Posse)over 7 years ago

Synchronous replication with milliseconds latency

Andriy Knysh (Cloud Posse)over 7 years ago

Many read replicas

btaiover 7 years ago

nice

btaiover 7 years ago

and costs? @Andriy Knysh (Cloud Posse)

btaiover 7 years ago

more expensive/same/cheaper than vanilla RDS?

Andriy Knysh (Cloud Posse)over 7 years ago

https://www.percona.com/blog/2018/07/17/when-should-i-use-amazon-aurora-and-when-should-i-use-rds-mysql/

Andriy Knysh (Cloud Posse)over 7 years ago

For production where you need bigger instances, the cost is relatively the same. With plain RDS you can get smaller and cheaper instances, but that are just good for testing and maybe staging

btaiover 7 years ago

are you using it for postgres or mysql

Andriy Knysh (Cloud Posse)over 7 years ago

Both

btaiover 7 years ago

we just talked to the solutions architect that said we can just restore the rds potgres snapshot to aurora and it will just work

Andriy Knysh (Cloud Posse)over 7 years ago

Yes

btaiover 7 years ago

did you find that to be true or did aurora change things

btaiover 7 years ago

nice

Andriy Knysh (Cloud Posse)over 7 years ago

Aurora changes things mostly in user and permissions management, and some other minor things

btaiover 7 years ago

mmm

btaiover 7 years ago

i am using alot of roles/databases in my rds instance

btaiover 7 years ago

multi-tenant so i wonder if that will change things

Andriy Knysh (Cloud Posse)over 7 years ago

E.g. even the master user you create is not the admin in the cluster

btaiover 7 years ago

thanks @Andriy Knysh (Cloud Posse)!

btaiover 7 years ago

i will test it out

Andriy Knysh (Cloud Posse)over 7 years ago

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Security.html

keenover 7 years ago

aurora's a bit more fun to codify setup, and to do instance up/downgrades (ie, you have to do each of them yourself. tip: just create new nodes in the cluster at the scale you want, then failover to them...)

keenover 7 years ago

if you make the mistake of scaling your write node, it will create an outage because aurora dont care.

keenover 7 years ago

oh, and I have a broke psql aurora node right now, it borked in prod, didnt failover. failed the cluster over, the node is still borked. can't add support to the account because no one can get into the vault in the office where the root acct's mfa key is.

keenover 7 years ago(edited)

(effing stupid that whole iam-users-can't-change-support)

Andriy Knysh (Cloud Posse)over 7 years ago

It’s even worse with regular Postgres and MySQL :)

keenover 7 years ago

regular rds used to be a dream to update/scale with maz turned on - punch it and walk away, it'd update the slave, then failover and update the master.

keenover 7 years ago

let it trigger in the maint window if you want

Alex Siegmanover 7 years ago(edited)

So looking at the reference architectures repository, there seems to be two accounts that seem to overlap:


root
The "root" (parent, billing) account creates all child accounts and is where users login.

Of note here is the "where users login." There's also an "identity" account:


identity
The "identity" account is where to add users and delegate access to the other accounts

I'm a bit unsure how this would look in reality. I'm not sure how you'd "login" to the root account, if your user is over in a separate account. Am I missing something? I thought in AWS your starting point always had to be wherever your IAM user existed, and from there you can assume roles in whatever fashion is needed.

Igorover 7 years ago

Does anyone have any experience with hosting images and videos that are optimal for each device? Is there an AWS service or an approach that's better than generating 10 versions of an image and using S3/Cloudfront?

Andriy Knysh (Cloud Posse)over 7 years ago

we recently deployed https://docs.aws.amazon.com/solutions/latest/serverless-image-handler/welcome.html. It uses http://www.thumbor.org/ to change image size/format/filter on the fly. Behind a CDN works ok and fast enough. It’s CloudFormation, not TF though

Igorover 7 years ago

Thank you

Igorover 7 years ago

Where can I find the CloudFormation template(s)?

Andriy Knysh (Cloud Posse)over 7 years ago

(click Next a few times) https://docs.aws.amazon.com/solutions/latest/serverless-image-handler/template.html

Igorover 7 years ago

🤦‍♂️ Sorry & thanks

chrismover 7 years ago

@Andriy Knysh (Cloud Posse) what are the running costs like? we currently use imageresizing.net on iis (m3 mediums) x3 with cloudfront over it as we're loading images from s3 etal

Andriy Knysh (Cloud Posse)over 7 years ago

https://docs.aws.amazon.com/solutions/latest/serverless-image-handler/overview.html

Andriy Knysh (Cloud Posse)over 7 years ago

As of the date of publication, the estimated cost for running the Serverless Image Handler for 1 million images processed, 15 GB storage and 50 GB data transfer, with default settings in the US East (N. Virginia) Region is as shown in the table below. This includes estimated charges for Amazon API Gateway, AWS Lambda, Amazon CloudFront, and Amazon S3 storage

Andriy Knysh (Cloud Posse)over 7 years ago

AWS Service	Total Cost
Amazon API Gateway	$3.50
AWS Lambda	$3.10
Amazon CloudFront	$6.00
Amazon S3	$0.23

chrismover 7 years ago

ta; id clicked architecture not clicking the overview was a page 😁

chrismover 7 years ago

Anyone experienced

* module.elasticache_redis.module.label.data.null_data_source.tags_as_list_of_maps: data.null_data_source.tags_as_list_of_maps: value of 'count' cannot be computed

recently starting
I ran the module a couple of weeks ago without an issue. Counts from beyond the grave

chrismover 7 years ago

ah https://github.com/cloudposse/terraform-null-label/issues/44

chrismover 7 years ago

🤔 is this only an issue because the enabled flag sets a count on a resource thats only ever 1 but terraform thinks it has an enumerable to work with that it cant

chrismover 7 years ago

the answer to that is no lol its in the label code 🤪

chrismover 7 years ago

tbh the elasticache module only needs a tags + id input; the additional dependency on label seems overkill; more injection > less dependencies

chrismover 7 years ago

of course, the dns parts of debatable use once you enable tls as the tls isnt configurable

raehikover 7 years ago

Hey all, I'm stuck in the world of IAM and I had a thought about permissions management. It's nice to split permissions into a user-role structure for management and auditing, but the way role assumption is done feels awkward, and users have to "know" what roles they have available to them. Does AWS provide a method to find out all the assumable roles for a given user?

Dave Yuover 7 years ago

is the fargate cli the closest thing in awsland to https://cloud.google.com/run/?

Erik Osterman (Cloud Posse)over 7 years ago

Probably

Erik Osterman (Cloud Posse)over 7 years ago(edited)

btw, there are (2) clis for AWS

Erik Osterman (Cloud Posse)over 7 years ago

Also, they've started developing this one again: https://github.com/jpignata/fargate

Erik Osterman (Cloud Posse)over 7 years ago

new maintainer

Igor Rodionovover 7 years ago

Does anyone knows how RDS encryption storage works with KMS key that have rotation enabled?

Andriy Knysh (Cloud Posse)over 7 years ago

https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html

Alex Siegmanabout 7 years ago(edited)

This just popped in to my email: https://aws.amazon.com/app-mesh/

I'm wondering if this could also integrate with say GKE for multi-cloud application networking. I also wonder how that integrates with EKS, since I've seen envoy in use primarily as a app mesh for K8S

btaiabout 7 years ago

anyone know if we can use different encryption keys for a single database storage in an RDS instance? (multi-tentant rds instance)

chrismabout 7 years ago

https://github.com/godaddy/kubernetes-external-secrets

mumoshuabout 7 years ago

@mumoshu has joined the channel

Igorabout 7 years ago(edited)

Does anyone know of a good way to tag shared resources for billing reporting/monitoring purposes? For example, if I have an ALB that's in front of two web apps - W1 and W2, can I have a billing report that includes 1/2 of the ALB cost with W1 and the rest with W2?

Erik Osterman (Cloud Posse)about 7 years ago

@Igor are you using Kubernetes by anychance?

Igorabout 7 years ago

No, not using Kubernetes

vishnu.shuklaabout 7 years ago

hey all, I am stuck at AWS code build with ruby framework, seeking help on this

Tim Maloneabout 7 years ago

post what you’re stuck on - someone might be able to help 🙂

vishnu.shuklaabout 7 years ago

there must be issue with the buildspec.yml file or image

vishnu.shuklaabout 7 years ago

i tried with many other option as well

vishnu.shuklaabout 7 years ago

everytime i get different error

xluffyabout 7 years ago

exit status is 127

xluffyabout 7 years ago

It means command not found

vishnu.shuklaabout 7 years ago

its like on of the error

xluffyabout 7 years ago

see above line, doesn’t have sudo command

vishnu.shuklaabout 7 years ago

does AWS code build provide docker image with mysql installed on this?

vishnu.shuklaabout 7 years ago

@xluffy do you have any buildspec.yml file for Ruby to use on AWS code pipeline.

vishnu.shuklaabout 7 years ago

here is the latest error i am getting

xluffyabout 7 years ago

see error, this is another error, in this container, doesn’t any JS runtime (u need a container with js runtime)

vishnu.shuklaabout 7 years ago

provided AWS docker image doesn't have it , i tried installing it but no success, also AWS code build has only single image for Ruby

nutellinoitabout 7 years ago

Hello, anyone using ec2 spot fleet plugin with jenkins?

vishnu.shuklaabout 7 years ago

can someone help me one this.

Exequiel Barrireroabout 7 years ago(edited)

Just in case anyone find it useful.

AWS Management Console down ⚠️ without region #aws #outage

(https://status.aws.amazon.com/)
• (link: https://us-west-2.console.aws.amazon.com/console) us-west-2.console.aws.amazon.com/console works

• (link: https://us-east-2.console.aws.amazon.com/console) us-east-2.console.aws.amazon.com/console works

• (link: https://us-east-1.console.aws.amazon.com/console) us-east-1.console.aws.amazon.com/console does NOT work ❎️
(link: https://console.aws.amazon.com/console/home) console.aws.amazon.com/console/home 😞

So you can basically by-pass the error specifying the console region in the access url.
To hit a specific service in us-east-1 you can use the service URL, eg: https://us-east-1.console.aws.amazon.com/ec2/v2/home?region=us-east-1#Home:

chrismabout 7 years ago

When you accidentally spin up a k8 cluster using T instances and the cpu-burst wipes out leaving 1 node running at ~20% of cpu

chrismabout 7 years ago

https://media.giphy.com/media/yfEjNtvqFBfTa/giphy.gif

Lee Skillenabout 7 years ago

All kinds of awesome:
https://infrastructure.aws/

rohitabout 7 years ago

Is there a way to get objects from AWS S3 private bucket using cloudfront without presigned URL’s ?

Lee Skillenabout 7 years ago

Yup, you can use an Origin Access Identity on the CloudFront distribution which has access to the S3 bucket via a policy. If you need your own auth at the CDN level you can implement it with a small Lambda @ Edge too. :)

rohitabout 7 years ago

@Lee Skillen thanks for answering my question

rohitabout 7 years ago

what would be advantage of using Origin Access Identity over presigned URL/cookies ?

Lee Skillenabout 7 years ago(edited)

It's transparent to those using the CDN for a start, and it means you don't need to generate presigned URLs upfront. The downside is that you might still be interested in protecting the content, so need to think about auth in a different way.

Lee Skillenabout 7 years ago

What kind of content is it?

rohitabout 7 years ago

it is all media files

Lee Skillenabout 7 years ago

Do you need auth? What if someone obtains a URL and distributes it to others?

Andriy Knysh (Cloud Posse)about 7 years ago

if all content of the bucket is not a secret, then having private or public bucket does not make any difference since your users will see all the files via CloudFront. With a private bucket, use origin access identity as @Lee Skillen mentioned

Andriy Knysh (Cloud Posse)about 7 years ago

https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/blob/master/main.tf

rohitabout 7 years ago

we want to make the media files private, so planning to use private bucket

Lee Skillenabout 7 years ago(edited)

Some small difference is that a public bucket can be listed but not necessarily true with CloudFront. It may make a difference if you don't care if files are public, but also don't want easy access to all of the other files. It wouldn't make sense for me, but I have seen people do this with URLs that are impossible to guess upfront (e.g. with randomised prefixes or something else in the URL). But if you go to that extent, I would just throw an auth method in there via a Lambda @ Edge. :)

Andriy Knysh (Cloud Posse)about 7 years ago

I have seen people do this with URLs that are impossible to guess upfront

Andriy Knysh (Cloud Posse)about 7 years ago

that’s security by obscurity 🙂 bad idea, never works

Lee Skillenabout 7 years ago

Depends on your goal, but I broadly agree :)

rohitabout 7 years ago

i haven’t used cloudfront + Lambda Edge, did you use any auth method with Lambda Edge ?

Igorabout 7 years ago

Is there a way to automatically update Amazon AMIs on a launch template to the latest, instead of having to rerun terraform against it on monthly basis?

Lee Skillenabout 7 years ago

@rohit It depends how fancy you want to get. How are users authenticated before they access the CDN? I assume you wouldn't want them to have to pass a username/password via basic auth if they were already authed before? In fact, are they authed at all or "anonymous"? Is the CDN on a subdomain of your main app website (if any)? You said media before, is it for static assets, downloads or streaming? Lots of questions and possibilities. :)

March 2019 Browse by date May 2019