#aws - August 2021

Not sure if this is the right place to post but I am having a heck of a time trying to figure out why my Lambda is is not getting triggered by MSK. When I setup the trigger I end up getting this error after a bit of time: PROBLEM: Connection error. Please check your event source connection configuration. This error is not super helpful so I am wondering if there is another spot to get more details. I have tried a number of things including trying to run the lambda in the same VPC as my cluster but no luck. This could be related to how the cluster is setup but wonder if anyone has any pointers on where to get some more troubleshooting logs or info (also, my lambda does nothing, just logs right now cause I want to make sure everything is wired up first).

Hi there, I was wondering if I could duplicate traffic with ALB. we want the same traffic that goes to production also goes to the staging ECS target group as well. Is there a way to go around this?

https://github.com/jpillora/chisel

Exploring this reverse tunneling tooling in chisel project and wrapping my head around this and how it would work with a containerized infrastructure with ECS Fargate.

Is anyone using Docker containers for a service that assigns random ports for each open session?
I really want to stick with ECS Fargate if I can, but with several hundred ports maintaining open sessions and passing traffic though, I'm not certain yet that it will support that without issues.

On a high level, I'm assuming (correct me if you disagree), if a server needs to maintain open sessions with each = unique random port, I'm going to have to stick with a normal EC2 Server most likely if scaled beyond a couple tunnels.

I have an issue with the HTTPS Listener on ALB not been updated when the Green/Blue Deployment is done. Since this uses two target groups to handle the traffic shift, it automatically changes the HTTP listener to the replacement task while the HTTPS remains on the Green(Old) target. in the case the ALB to return 503 error since both Listener is not on the same target groups. To solve this I have to go to the HTTPS listener and change it to the replacement task target group. Is there a way I can get HTTPS to listen to the deployment and update the target group itself as the HTTP Listener does?

I want to check an interpretation of "least-privilege" in AWS IAM, and see if others interpret this differently.  If someone were to ask me to design a policy that is "least-privilege", I would think the more exact the Actions and Resources, the better.

    {
        "Sid": "",
        "Effect": "Allow",
        "Action": [
            "lambda:*",
            "events:*"
        ],
        "Resource": [
            "arn:aws:lambda:region:123456789012:rule/my-pattern-*",
            "arn:aws:lambda:region:123456789012:layer:*",
            "arn:aws:lambda:region:123456789012:function/my-pattern-*"
        ]
    }

Is this accurate or is "least-privilege" more so on the Actions that are in the policy and not the resources?

{
        "Sid": "",
        "Effect": "Allow",
        "Action": [
            "s3:PutObject",
        ],
        "Resource": [
            "arn:aws:s3:::bucket-name/*",
        ]
}

https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege
I can't really tell from the best practices page, what is actually "good enough". Or is it actually, resources would ideally be specific arns only?

I walked into a sticky situation regarding SSO with multiple AWS accounts, roles, and external groups that's tied into AD. There is a group per role per team per account or something to that extent. I spun up a mgmt account and plan on making a role that looks like <Prod/nonProd>-<team>-<role> so we'd have nonProd-SRE-admin, nonProd-SRE-readOnly, Prod-SRE-admin, and Prod-SRE-readOnly for starters. I don't think that's scalable beyond 4 or 5 teams, but it's a huge improvement over 100+ groups. Any advice on a better short-term solution? I want to use some of the CloudPosse modules because it's going to take a lot of effort to fix some of this.

Is anyone else having issues with yum on amazonlinux? http://amazonlinux.default.amazonaws.com/2/core/latest/x86_64/mirror.list is returning a 403 which seems not-correct

heads up that us-east-1 is having issues with AMI filtering incorrectly classifying images as ‘paravirtual’ which breaks a lot of things if you are trying to use ENA

support says this is “impacting other services” too

Hello, I am trying to create a msk cluster with the module (https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster) . When i provide the number_of_broker_nodes = 2 with two subnet ids (different AZs) . It is expected to create 4 broker in the cluster . but the module is creating only 2 . If i change the number_of_broker_nodes = 3 or more then i am getting the error ,

Error: Invalid index

  on .terraform/modules/kafka_msk.kafka_msk/main.tf line 158, in module "hostname":
 158:   records = [split(":", local.bootstrap_brokers_combined_list[count.index])[0]]
    |----------------
    | count.index is 2
    | local.bootstrap_brokers_combined_list is list of string with 2 elements

The given key does not identify an element in this collection value.


Error: Invalid index

  on .terraform/modules/kafka_msk.kafka_msk/main.tf line 158, in module "hostname":
 158:   records = [split(":", local.bootstrap_brokers_combined_list[count.index])[0]]
    |----------------
    | count.index is 3

Has anyone encountered this before and have any workaround for it ? Thanks in advance

Is it possible to request Pinpoint SMS phone numbers via API? I cannot find any documentation.

Hi,
I am using "terraform-aws-eks" & "terraform-aws-eks-node-group" modules.
Can I have examples to add multiple node groups, which have different settings like name, instance type, desired_size, private vs public?

Hi Champs, can anyone help with k8s deployment script for ingress with tls on fargate eks ..thanks

Is there an easy/efficient way to force destroy all resources in an AWS account (without destroying the account itself)? In this case, infra is managed by terraform. Often times we run into dependency issues (especially around networking resources) that result in failed terraform destroy attempts.

[Blog Post] https://blog.cloudyali.io/absolute-minimum-every-developer-must-know-about-aws-security

I’m running two aurora postgres workloads in US-WEST-2 region:

cluster Regional cluster	Aurora PostgreSQL	us-west-2	2 instances	    Available
qa-0    Writer instance	    Aurora PostgreSQL	us-west-2a	db.t3.medium	Available
qa-1    Reader instance	    Aurora PostgreSQL	us-west-2b	db.t3.medium	Available

and yet my console reports:

Aurora is not available in this region (US West (Oregon)). Use the region selector to switch to a supported region.

Have you ever seen something like this? 🤔

EC2 is out of capacity for m6g.xlarge in availability zone us-east-1a

fyi - use1-az2

Hi colleagues, anyone is aware of any article that explains how launch configuration and auto scaling groups are being used by EKS?

👋☺️ Hi All!! Based on ease of administration… what do you recommend for VPN server in hub and spoke model in AWS across multiple AWS children accounts?

1. OpenVPN AMI
2. OpenVPN via Helm Chart
3. AWS VPN Client

Hello Team, Can anyone help me how to pass multiple cidrs to security group AWS terraform module - https://github.com/cloudposse/terraform-aws-security-group
i am trying to pass it like this, it seems terrafrom is crashing

module "sg" {
  source = "cloudposse/security-group/aws"
  attributes = ["primary"]
  allow_all_egress = true
  rules = [
    {
      key         = "ssh"
      type        = "ingress"
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = concat(
        module.global.le_egress_cidr,
        module.global.shared_services_cidr,
        module.global.ibm_egress_tdq
      )
      self        = null
      description = "Allow SSH from company egress"
    }
  ]
  vpc_id  = module.vpc.vpc_id
  context = module.label.context
}

hello. i've been implementing https://github.com/cloudposse/terraform-aws-datadog-lambda-forwarder.git in our commercial and govcloud accounts. for the most part, everything on the commercial side is working, but when trying to implement on the govcloud side, i'm getting

│ Error: Error creating Cloudwatch log subscription filter: InvalidParameterException: Could not execute the lambda function. Make sure you have given CloudWatch Logs permission to execute your function.
│ 
│   with module.monitoring.module.monitoring_common.module.datadog_forwarder.module.datadog_lambda_forwarder.aws_cloudwatch_log_subscription_filter.cloudwatch_log_subscription_filter["vpclogs"],
│   on .terraform/modules/monitoring.monitoring_common.datadog_forwarder.datadog_lambda_forwarder/lambda-log.tf line 150, in resource "aws_cloudwatch_log_subscription_filter" "cloudwatch_log_subscription_filter":
│  150: resource "aws_cloudwatch_log_subscription_filter" "cloudwatch_log_subscription_filter" {

i see no difference in the policies and wondering if this is something unique to govcloud.

this is due to more hardcoding of the aws partition

submitting a bug report and PR