#aws - March 2021 | Slack Archive

81 messages

Discussion related to Amazon Web Services (AWS)
Archive: https://archive.sweetops.com/aws/

Feb 2021

March 2021

Apr 2021

PePe Amengualalmost 5 years ago

What is the latest greatest ECS blueGreen deployments in AWS? I have used CodeDeploy with multiple ASGs before and looked at App Mesh too but I wonder what is the easiest these days? ( weighted TGs?)

Troy Tailleferalmost 5 years ago

I am trying to make a kinesis autoscaler lambda based on existing code basically update shard count based on incoming records alarm metric. During testing I notice something odd in using aws cli commands to get the number shards shown above. Basically describe-stream-summary says the OpenShardCount is one this seems like the right answer but describe-stream and list-shards report there are 4 shards. Which is correct ? Why are they not consistent ? Hope there is a kinesis expert here who can explain what is going on thanks

Pavelalmost 5 years ago

i have CF with S3 origin, the origin has origin_path = "/build", CF has its first behavior as "/url/path/*".
I get the The specified key does not exist error and the Key ends up being /build/url/path/index.html

Pavelalmost 5 years ago

I can access the files from root of the cdn but not from my path pattern

Pavelalmost 5 years ago

do i have to have the origin folder structure (s3) match my behavior path?

Alex Jurkiewiczalmost 5 years ago

Yes

Pavelalmost 5 years ago

this client jenkins s3 plugin does not allow me to do that

Alex Jurkiewiczalmost 5 years ago

You could rewrite the path with a lambda@edge function 🤷‍♂️

Pavelalmost 5 years ago

thats where im at now 🙄

Pavelalmost 5 years ago

still need a lambda because if its in the folder it has no idea what to do with index.html

RBalmost 5 years ago

can you folks use threads plz

Pavelalmost 5 years ago

its fine im done

Bart Coddensalmost 5 years ago

I am a bit puzzled by a network issue

Bart Coddensalmost 5 years ago

Machine has two firewall groups assigned: outbound = all open, inboud = ssh open from my ip

Bart Coddensalmost 5 years ago

I can access ssh from my workstation

Bart Coddensalmost 5 years ago

Making a ssh connection FROM the instance does not work

Bart Coddensalmost 5 years ago

when I tcpdump my traffic, I can see traffic going out of the machine

Bart Coddensalmost 5 years ago

ha I found it, the configuration of the firewall group changed a bit

Marcin Brańskialmost 5 years ago

Woooohooo! So simple and now it’s there

I shouldn’t be that much happy for it but everytime I setup ELK on AWS (soo many times) I check if it’s available and here it is.

Amazon Elasticsearch Service now supports rollups, reducing storage costs for extended retention

Takanalmost 5 years ago

hi guys,
anyone knows how to create "trusted advisor" in terraform?

Ofir Rabanianalmost 5 years ago

Hi everyone,
Let’s say that I have a terraform setup with an rds instance. After a while, I want to restore to a given point in time through a snapshot that i’m creating every day. Given that AWS limits to restoring the snapshot to a NEW instance, how can I still control this new instance using terraform? what’s the correct process to have here?

RBalmost 5 years ago

shamelessly asking for upvotes here https://github.com/99designs/aws-vault/pull/740

tldr, we figured out a way to plist the aws-vault --server https://gist.github.com/nitrocode/cd864db74a29ea52c7b36977573d01cb

MrAtheistalmost 5 years ago

Anyone know why AWS doesnt have a default iam policy for "ecs read only"? i have to create one just for this... ¯\(ツ)/¯

msharma24almost 5 years ago

Hi Guys - Is there a way to get the AWS Organisation ID (unique identifier)) via AWS CLI / API ?

michaelssinghalmost 5 years ago(edited)

I am running into some difficulties with provisioning a Windows EC2 instances with a Powershell script, which is passed into a aws_launch_configuration as such:

user_data_base64 = base64encode(data.template_file.powershell.rendered)

The script is also quite simple, it downloads a .exe from the internet and then starts a silent install with Start-Process:

<powershell>
Start-BitsTransfer -Source ...
Start-Process ..
</powershell>

This is my first time working with Powershell and provisioning Windows EC2s so I may be missing something but when I RDP into the machine the executable is neither downloaded nor installed.

michaelssinghalmost 5 years ago

If paste the contents of the Powershell script into Powershell on the instance, it works as expected however.

Patrick Jahnsalmost 5 years ago

Does anyone have a list of common dns names for aws services? I am trying to get a feeling for their patterns

Steve Wade (swade1987)almost 5 years ago

can anyone help with ☝️ please its like RDS is partially completed the upgrade from 5.6 to 5.7

Steve Wade (swade1987)almost 5 years ago

is there a way to force the pending modifications now instead of waiting for the maintenance window?

Alex Jurkiewiczalmost 5 years ago

yes, 'apply immediately'

Alex Jurkiewiczalmost 5 years ago

it's an option you can pass when modifying an rds instance/cluster. Either via web console or api

Steve Wade (swade1987)almost 5 years ago

i am trying to set that now

Steve Wade (swade1987)almost 5 years ago

but it don't let me set it

Steve Wade (swade1987)almost 5 years ago

i can see the pending modifications via the API but can't seem to apply them

Alex Jurkiewiczalmost 5 years ago

are you passing apply immediately and a change to the config? You can't pass only 'apply immediately' with no changes

Steve Wade (swade1987)almost 5 years ago

i don't want to make any changes though i want it to apply the pending mods e.g. upgrade to 5.7

Alex Jurkiewiczalmost 5 years ago

you need to re-submit the pending modification with apply_immediately set to true

Alex Jurkiewiczalmost 5 years ago

when you submit a change with that flag, all pending modifications are immediately applied

Steve Wade (swade1987)almost 5 years ago


An error occurred (InvalidParameterCombination) when calling the ModifyDBInstance operation: Current Parameter Group (de-prd-yellowfin-01-20210219152210698600000003) is non-default. You need to explicitly specify a new Parameter Group in this case (default or custom)

Steve Wade (swade1987)almost 5 years ago

think i may have fixed it

PePe Amengualalmost 5 years ago

Has anyone enforced IMDSv2 on their instances and had problems with cloud-init not starting?

Vlad Ionescu (he/him)almost 5 years ago(edited)

FYI, https://pages.awscloud.com/pi-week-2021 is happening! It will be a fun one 😉 A bunch of S3 + Data in general + some Serverless

Alex Jurkiewiczalmost 5 years ago

I want to host a single file over HTTPS with a custom domain. Is there a simpler solution than S3 bucket + CloudFront + ACM cert? Simpler meaning serverless, no ec2 + nginx in user-data solutions 😛

Zachalmost 5 years ago

someone appears to have published and then retracted this post (but it popped on my AWS News RSS), so I think we’re going to see Fargate exec tool soon!
https://aws.amazon.com/about-aws/whats-new/2021/03/amazon-ecs-now-allows-you-to-exec[…]commands-in-a-container-running-on-amazon-ec2-or-aws-fargate/

Zachalmost 5 years ago

now its up
https://aws.amazon.com/blogs/containers/new-using-amazon-ecs-exec-access-your-containers-fargate-ec2/

Walialmost 5 years ago

Interesting ended up git cloning requests when virtualenv didn’t work. Anyone encountered this before?

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'requests'END RequestId: 7ed24ad6-1b95-4600-9a35-d379726f6b47

Mohammed Yahyaalmost 5 years ago

https://github.com/salesforce/cloudsplaining

Maarten van der Hoefalmost 5 years ago

new manager with retail background I guess

lorenalmost 5 years ago

does anyone write their lambdas such that they understand a common, fake "test event"? such that you can invoke it with that event just to validate that the package really has the imports it needs?

kalyan Malmost 5 years ago

Hi guys what are some top most/Must use tools in Managing the Kubernetes on aws eks or other clusters. any recommendations on best Practices?

Maciek Strömichalmost 5 years ago

Anyone using Cognito’s MFA functionality? How do you block ability to disable previous MFA setup by calling associate software token again and again? this call can be sent by anyone (it just requires a valid access token) and if it’s sent for the second time it automatically overrides the previous setup and disables mfa on login.

RBalmost 5 years ago(edited)

the author provided some feedback against it. if anyone is interested in daemonizing aws-vault using launchd, please leave some feedback.

aws-vault: Start metadata server without subshell (non-daemonized)

Ikanaalmost 5 years ago

Is it possible to contract cloudposse’s services through an AWS marketplace private offer?

Matt Gowiealmost 5 years ago

Hey folks — Is anyone using an external Security Saas product like Fugue or other to replace using AWS Config / SecurityHub? AWS account rep is suggesting we utilize https://www.fugue.co/ and I’d be interested in hearing folks thoughts.

MattyBalmost 5 years ago

How does everyone handle MFA for root credentials for your AWS accounts (or whatever). Someone had the idea to just use an OTP device and store it in the safe, but that will take 2h+ for anyone local, and if you're in another state then you're screwed. A workaround would be to just open a case with amazon to reset MFA which we're fine with. Search wasn't super helpful...help, por favor!

Darren Cunninghamalmost 5 years ago

Anybody using

and figured out a way to consolidate API Gateway logs? Currently each requests creates 28 log messages. Creating 28 million log messages per million requests is silly. Not a ghastly expense, but one that I'd like to mitigate.

michael sewalmost 5 years ago(edited)

Just curious if anybody has tried to do visualizations of AWS regions visually in something like PowerBI , grafana (or the AWS analog, quicksight?) PowerBI mentions ShapeMaps, but they need something called a shapefile or TopoJSON .. anybody tried this before?

Matt Gowiealmost 5 years ago

Anyone experiencing DNS issues with the AWS Console today?

michael sewalmost 5 years ago

stupid CLI question: I'm creating an SSH key and want to tag my resources in the same commmand. I do with with the --tag-specifications flag.

aws ec2 create-key-pair  \
  --key-name bastion-ssh-key  \
  --tag-specifications 'ResourceType=key-pair,Tags=[{Key=deployment:environment,Value=sbx},{Key=business:steward,Value=my@memail.com},{Key=security:compliance,Value=none}]'
. . .

How can I split the tags into multiple lines per tag? I've tried a few different ways and the CLI keeps complaining to me. Seems you have to put this in a single line. Not opposed, but this just makes it unreadable.

managedkaosalmost 5 years ago

@michael sew you may want to consider creating a json template that you can import into the command, it will make things much neater! 😅

$ aws ec2 create-key-pair --generate-cli-skeleton
{
    "KeyName": "",
    "DryRun": true,
    "TagSpecifications": [
        {
            "ResourceType": "snapshot",
            "Tags": [
                {
                    "Key": "",
                    "Value": ""
                }
            ]
        }
    ]
}

managedkaosalmost 5 years ago

then:

aws ec2 create-key-pair --cli-input-json FILE

Darren Cunninghamalmost 5 years ago(edited)

Anybody have an example of a WAF v2 Rule that blocks requests with an http protocol? I'm figuring that I'm looking for SingleHeader but not sure if I should be looking for protocol , http.protocol or X-Forwarded-Proto or if I'm totally off base

Mohammed Yahyaalmost 5 years ago

https://aws.amazon.com/blogs/developer/introducing-aws-sso-support-in-the-aws-toolkit-for-vs-code/

Mohammed Yahyaalmost 5 years ago(edited)

AWS Gurus: How can I read secrets from AWS Secrert manager inside EKS pod ?

lorenalmost 5 years ago

nice, this might make step functions rather a lot easier to define and use... https://aws.amazon.com/about-aws/whats-new/2021/03/aws-step-functions-adds-tooling-support-for-yaml/

managedkaosalmost 5 years ago

👀 🤔
https://www.youtube.com/watch?v=1pnMsWiM3P8&ab_channel=RKVSRaman

managedkaosalmost 5 years ago

anyone tried it yet?
https://aws-workbench.github.io/

Mahmoudalmost 5 years ago

Would anyone happen to have come CLI command for getting all load balancers with target groups with no listeners?
We’re looking to clean up our dangling LBs, but I’m not super experienced with AWS CLI
https://www.cloudconformity.com/knowledge-base/aws/ELBv2/unused-load-balancers.html#
I’ve mostly been following the steps here, but I’m attempting to create some kind of command to get all the ones with no target instances

managedkaosalmost 5 years ago(edited)

@Mahmoud this worked for me:

echo -e "LoadBalancer\tListenerCount"

for i in $(aws elbv2 describe-load-balancers --query="LoadBalancers[].LoadBalancerArn" --output=text);
do
    echo -e "$(echo ${i} | cut -d/ -f 3)\t$(aws elbv2 describe-listeners --load-balancer-arn=${i} --query='length(Listeners[*])')"
done | tee report.txt

Fabianalmost 5 years ago

Hi. We have a daily process running, of which some jobs started failing about two days ago. Does anybody have an idea what might cause this? We've already followed the steps https://aws.amazon.com/premiumsupport/knowledge-center/batch-job-failure-disk-space/ The AWS Batch is failing with the error message “CannotPullContainerError: failed to register layer..: no space left on device”. This happens for only some jobs, not all.
I have already created a Launch template and given it a 500G storage and in user data, have set :

cloud-init-per once docker_options echo 'OPTIONS="${OPTIONS} --storage-opt dm.basesize=300G"'

Fabianalmost 5 years ago

Anyone?

Walialmost 5 years ago

Check thread @Fabian

Ashish Modialmost 5 years ago

Morning everyone!

I was wondering if it is possible to control recently announced auto tune feature for aws elasticsearch using terraform?
see here - https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/auto-tune.html

Bart Coddensalmost 5 years ago

Hi all, a external customer wants us to enable a s3 policy on a bucket:

Bart Coddensalmost 5 years ago

{
    "Version": "2012-10-17",
    "Id": "Policy1472487163135",
    "Statement": [
        {
            "Sid": "Stmt1472487132172",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::1234567:root"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::BUCKET_NAME/*"
        },
        {
            "Sid": "Stmt1472487157700",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::1234567:root"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::BUCKET_NAME"
        }
    ]
}

Bart Coddensalmost 5 years ago

I am a bit worried that they could open up the whole bucket as public

Bart Coddensalmost 5 years ago

but I am not sure, because this policy is locked to the specific principal

Stevealmost 5 years ago

We're seeing cloudwatch log group load time on console to fluctuate between 5-60+ secs. This started last couple of weeks. What's the most likely cause? Large number of log groups? Long retention? What's the level of performance that can be expected from aws for sub 1k log groups? What can be done to optimize performance?

managedkaosalmost 5 years ago

Hey team! Question:

TLDR: is there a good way to connect to a private AMQ without SSL?

Details:
I’m setting up an environment that uses Amazon MQ and I’d like to keep the service private (along with the rest of the resources). To that end, everything that needs to be private is sitting in a private subnet of the VPC: ECS, RDS, etc.

Because everything is private, I’m using the IP address to connect to the AMQ endpoint. However when i connect, the app fails with an SSL error:

cannot validate certificate for 10.0.2.29 because it doesn't contain any IP SANs

This leads me to think that the connection is SSL and the cert that AWS is serving up dosen’t have the IPs in it, but rather the DNS name of the MQ instance.

Googling for a solution, I found one doc that recommended putting an NLB in front of the AMQ and connecting to that but it seems (to me) that the connection might still fail; what about SSL validation between the ALB and the NLB? This solution also seems over engineered and potentially expensive given the addition of the NLB on top of the AMQ instance(s).
https://d2908q01vomqb2.cloudfront.net/1b6453892473a467d07372d45eb05abc2031647a/2019/09/10/Solution-overview.png

Anyway, just thought I would share this in the event someone has seen this type of issue before and knows a choose_one(good/reliable/affordable) solution for this problem. Cheers!

Alencar Junioralmost 5 years ago(edited)

I'm providing authorization to an API Gateway (proxy integration) with Cognito and I have a Lambda function (dockerized) requesting the API endpoint https://{id}.execute-api.{region}.<http://amazonaws.com|amazonaws.com> . I would like to know if it is possible to allow any resource within AWS including my dockerized lambda functions to access the API without authentication? Currently getting the response {"message":"Unauthorized"}
Note: That's a public API since I have external apps requesting it.

February 2021 Browse by date April 2021