#aws - March 2020

Hi, I would like to know if someone have an EKS cluster in any environment with only fargate implemented with the terraform-aws-eks-fargate-profile module.
I have tried to deploy a cluster with these module and the pods always remain pending. I even deployed with eksctl and I was comparing permissions of IAM, VPC, Tags etc ... and apparently it is the same on the AWS side, but with eksctl the pods work and with the terrafom modules not.

@Karoline Pauls why do you want to use ssh when you have system manager available?

I would avoid OpsWorks competelly, I've had the chance of using it and I can only describe it as a mess.

@Nikola Velkovski puppet 4 life! :D

Anyone able to help me with a CloudFormation/LandingZone issue? (description in thread)

has anyone got any direction on the tls issues surrounding bucket names with periods ?

the S3 deprecation plan https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/

Bucket Names with Dots – It is important to note that bucket names with “.” characters are perfectly valid for website hosting and other use cases. However, there are some known issues with TLS and with SSL certificates. We are hard at work on a plan to support virtual-host requests to these buckets, and will share the details well ahead of September 30, 2020.

For example, we cannot use the <https://company.bucket-name.s3.amazonaws.com/mypath> url if our bucket name containers a period. In this case the bucket name is company.bucket-name. If we use curl and its https link, it will fail unless we skip certificate validation using --insecure

Seems the version for this module went from 0.7.0 to 0.3.2 recently
https://github.com/cloudposse/terraform-aws-s3-bucket/releases

Is that the correct next version?

(also not sure where to post that question 👆️ )

Hello team. What do you suggest to fine tune permissions on AWS EKS pods ? I am planning to setup an multi-tenant AWS EKS and plan to use Namespaces for seperation but i need some advice I have found https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/ . I know AWS EKS does not have built-in fine tuned separation of duties. So what would be your suggestion ? I plan to hold each customer on separate namespace and only allow a service account to accesss to specific namespac

Also i am planning to use Fargate on this setup.

Or should i use https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html

When making requests to a private AWI GW through an internal NLB the request is routed to the correct stage but the stage name is also included in the resourcePath which breaks the request. This only happens when making the request thought the NLB.

hey, does anyone has ready to use cloudwatch alarms for kinesis firehose in cloudformation format e.g. for Delivery.DataFresshness, or ThrottledRecords or even better for those math expressions specified in https://docs.aws.amazon.com/firehose/latest/dev/monitoring-with-cloudwatch-metrics.html#firehose-metric-dimensions? (my friday lazy ass is asking (-: )

Not for that exact use but math expressions in CL alarms aye…

I wonder why they don’t include cw alarms in the docs of the resource.

e.g. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesisfirehose-deliverystream.html has a complete example with bucket, roles, policies and deliverystream

why they wouldn’t just add an example cw alarm in there?

ok, so for data freshness it should alarm only if 900 secs mark is crossed because that’s the maximum firehose limit

for anyone interested ;-))

  FirehoseDeliveryDataFreshness:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Firehose Delivery Data Freshness Alarm
      AlarmName: FirehoseDeliveryDataFreshness
      ComparisonOperator: GreaterThanThreshold
      Period: 60
      EvaluationPeriods: 3
      Threshold: 150
      Statistic: Maximum
      MetricName: DeliveryToS3.DataFreshness
      Namespace: AWS/Firehose
      Dimensions:
        - Name: DeliveryStreamName
          Value: !Ref FirehoseDeliveryStream
      TreatMissingData: breaching
      AlarmActions:
        - !Ref 'AlarmTopicArn'
      InsufficientDataActions: 
        - !Ref 'AlarmTopicArn'

my bufferinghints are set to 120 secs which means that on avg kinesis will deliver on the 121 second.

I gave it a littlebit more room

Hey all, I have a two questions regarding AWS Parameter Store:

1) Does anyone know of a good way to find when the last time a secret has been accessed?
2) Is there a good way to find how many times a given parameter has been accessed in a given time frame (ideally 24 hours)?

Anyone familiar with Firehose delivery to Redshift? From everything I can tell, it seems like your redshift needs to be publicly addressable (IE in a "public" subnet with an internet gateway) for Firehose to talk to it, complete with a /27 CIDR block range for Firehose to allow through the SG. Seems just weird to me to keep data stores publicly accessible like that. Is there any configurations out there where it can stay in the private subnet, but still get delivered to from firehose?

anyone have strong opinions on where ETL/ML structure should be set up? i currently have airflow in our staging k8s cluster but its kind of a pain to manage cross account permissions:
• could move airflow to our prod k8s (worried about doing this)
• could create a new k8s cluster in prod account and move it there
• or could create a new aws data account that has access to everything?
any insights/experiences are appreciated

https://github.com/bottlerocket-os/bottlerocket -> AWS's container OS looks interesting. Has an update strategy that reminds me of upgrading F5 LTMs....

https://aws.amazon.com/about-aws/whats-new/2020/03/amazon-aurora-with-postgresql-compatibility-supports-amazon-aurora-global-database/ niiice

Hello, Im looking for . modules that deploy eks with s3 backend + dynamodb for lock state

Hi Guys, I am new to this channel and trying to get some advice on the rolling update EC2 on the ASGs. I am trying to use this module https://registry.terraform.io/modules/cloudposse/ec2-autoscale-group/aws/0.4.0 and seeing that everytime I update userdata, instance type it just creates a new version of launch template but doesnt do any rolling update on the ASG. Is there is any sort of workarounds available as discussed in https://github.com/hashicorp/terraform/issues/1552

Morning everyone ! Quick VPC/EC2 question. When you change the Route Table for a subnet, is it required to restart/launch new EC2 instances on that subnet to reflect the change ? Is it immediate ?

I'm trying to sign a request on an ECS task to sts using my Instance profile IAM Role. When I check the IAM Request headers on my requests, it looks like the session token for the instance role is not being included.

Is there a way to get the current session token on the task?

How do people here lower replication lag in aurora replicas ? I'm looking at write operations and see a direct correlation with spikes in replication lag which is expected but the spikes are well over 100 ms reaching around 1 sec

Hello People !! I have a simple problem and I would like to know if anyone here have worked around it. I need to know , from within an EC2 instance, what is the Target Group that the instance is attached to, and then be able to de-register from it…

I just don’t want to reinvent the wheel in case someone has a script for this 🙂

Thanks a lot !

Hey,

can I somehow see which instances are out of stock for a specific aws region? we have been unable to launch c5.xlarge in eu-central-1a a couple of times this week and I am trying to find a source that will tell me when something is unavailable

Hey guys! I have an ECS service running my api in containers and today I noticed that the running containers stopped after 24 hours. It happened on staging as well on production today after exactly 24 hours, with the events separated excatly by time the deployment from staging to production took.
Is this expected behavior?

No, that doesn't sound right. I've just checked and have Fargate tasks running since January.

Is chamber able to read secrets that it didn't write? I have set the KMS alias, and I can list the keys, but they are showing up with Version 0, and I am unable to read them.

Adding @U010XGY9B46 bot

With RDS IAM authorizations for database users, is it possible to have ‘real users’ (ie, Tom Jones) who log into AWS via Okta SAML and assume a common role (“Developer”) still have unique RDS IAM authorizations? ie TomJones has okta username tJones@myco and assumes the Developer role, I still want him to use a database IAM user tJones, and not have access to the ‘bSmith’ database IAM user that maps to user Bob Smith.

This message was deleted.

Anyone have thoughts on only using Workmail to forward mail to another address? Im moving my domain away from GoDaddy to R53. I currently have my email setup to forward to my gmail address and then when I send from there gmail is set to make it look like im sending from my domain address.
Can I use Workmail as a "proxy" the same way and have it dump all my email to my gmail address? My other option is to just go with Gsuite basic plan and basically pay $6 a month to do the same thing.

I do this, I think ? 🙂

Anyone have any ideas on how to setup any type of Directory Service (ie. AWS Managed Microsoft AD) in the Ningxia (China) region WHILE YOUR SOURCE DIRECTORY LIVES IN VIRGINIA. The documentation I’m reading is pretty much saying you need your source directory in China as well. I haven’t been able to Google this…

Hey folks — I’ve got a fun client problem I’m trying to solve. Looking for some input if anybody has a good idea…

I built the environment like so:

3 domains => CloudFront distribution => ALB => ECS Service

This works great, the ECS Service webserver handles providing different views of the application depending on the domain name via the Host header. That in combination with a couple ALB Listener Rules does the trick for what each of those domains are expected to show.

One of the domains however is the “Admin CMS” domain. I just found out that the client would like to IP Whitelist this domain so only a couple VPNs can access it to help further lock down access to the Admin CMS. This is tricky of course since I cannot use standard security control mechanisms (the ALB security group or VPC Network ACLs) since ALL domains are routed through the same ALB, which means I cannot apply a security group rule to only one domain.

So I’m looking for a workaround… Maybe through an additional ALB Listener Rule? Has anyone used Listener Rules to block / allow CIDRs? Is that a thing? Looked into this shortly yesterday and I’m about to start researching more now, but figured I’d ask here in case somebody goes: “Oh yeah of course, do X”.

I could obviously spin up another ALB, point the domain at that, and then use that ALB’s security group to control access. But unfortunately that increases cost and complexity of the system which I’m trying to avoid.

Any suggestions?