#office-hours - August 2022

I'm just embarking on 'decomposing' a small number of shared/'monolithic' AWS accounts into a larger number of focussed AWS accounts. Naming is hard. Feels like every account/workload/stage needs an expressive string to embed in its resource names, so when you're referencing those resources across accounts, it's easy to see what's what without needing to parse account IDs. That string needs to be short (because of the many resource name length restrictions), expressive (so it actually helps) and unique (for facepalm avoidance). Am I thinking about this right? Any advice?

I would like to discuss this today https://github.com/cloudposse/terraform-aws-efs/issues/114

and also there is a nice number of replication supported with Terraform now, which will help set up a DR easily:
• KMS
• ECR
• SecretManager
• EFS
• RDS

Regarding GitOps ...

Without going into too much detail, I use Github actions and a structure of application and env repositories. Applications publish charts and terraform modules. Env repositories use those in addition to a env specific configuration. Everything is automated, the infrastructure and Kubernetes resources state is located as code in the env repositories. There is a lot of reusability with GH actions, there are a few cool custom features like automatic conversion of terraform output to helm values, automatic blue/green deployments for k8s version upgrades and others. Because of GH actions I don't have to deploy/manage/troubleshoot/fix anything myself, I don't have to worry about scalability. Multi-tenancy is achieved by creating additional env/tenant repositories and updating configuration.

Questions are:

Are solutions like Flux/Argo worth the self managing/scaling/troubleshooting/migration efforts?
Can those solutions handle terraform? Or would I need another service like Spacelift in addition to Flux/Argo?
Anybody using Flux/Argo? Pros/Cons?
Anybody migrated from GH to Flux/Argo?

Sorry for the general open ended questions, I know that in the end I will have to decide for myself but I was curious about your thoughts and experience. Maybe this is something for the Office Hours. I will also go through older episodes try to find GitOps content.

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Ralf Pieper has joined Public "Office Hours"

Chris Barnes has joined Public "Office Hours"

Dave Gregory has joined Public "Office Hours"

Linda Pham (Cloud Posse) has joined Public "Office Hours"

Vlad Ionescu has joined Public "Office Hours"

Christopher Pieper has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Roy Sprague has joined Public "Office Hours"

Zadkiel has joined Public "Office Hours"

Ian Bartholomew has joined Public "Office Hours"

Tim Gourley has joined Public "Office Hours"

Bradley Peterson has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

I am interested in testimonials about Oracle Cloud.

Martin Palastanga has joined Public "Office Hours"

Adam Buggia has joined Public "Office Hours"

Josh has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Charles Smith has joined Public "Office Hours"

Zadkiel has joined Public "Office Hours"

Andrew Vitko has joined Public "Office Hours"

Ashwin Jacob has joined Public "Office Hours"

Antarr Byrd has joined Public "Office Hours"

Ben Smith (Cloud Posse) has joined Public "Office Hours"

Guilherme Borges has joined Public "Office Hours"

Ozzy has joined Public "Office Hours"

Vincent Werner has joined Public "Office Hours"

tamsky has joined Public "Office Hours"

Michael Williams has joined Public "Office Hours"

Yusuf has joined Public "Office Hours"

Naiman Daniels has joined Public "Office Hours"

Sherif Abdel-Naby has joined Public "Office Hours"

Azar AKB has joined Public "Office Hours"

Marc Tamsky has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Paul Bullock has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

13153275398 has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

13153275398 has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

Sherif Abdel-Naby has joined Public "Office Hours"

Links from today's session:
https://medium.com/tinder/exploiting-github-actions-on-open-source-projects-5d93936d189f
https://github.com/TinderSec/gh-workflow-auditor
https://beta.sayhello.so/
https://github.blog/changelog/2022-08-03-github-actions-remove-offline-self-hosted-runners/
https://github.com/runatlantis/atlantis/pull/997
https://aws.amazon.com/blogs/security/how-to-eliminate-the-need-for-hardcoded-aws-credentials-in-devices-by-using-the-aws-iot-credentials-provider/
https://aws.amazon.com/blogs/iot/securely-ingesting-large-sized-payloads-from-iot-devices-to-the-aws-cloud/
https://docs.aws.amazon.com/iot/latest/developerguide/iot-provision.html
https://github.com/cloudposse/terraform-aws-efs/issues/114

Discussion topic : I have the privilege of designing the VPC’s for my org and i’m looking for insights. What would you do differently if you had this luxury (e.g use IPAM e.t.c)? Should I go multi-account with multi-vpc or embrace the touted simplicity of Shared VPC?

AWS Storage Day is happening: https://aws.amazon.com/blogs/aws/welcome-to-aws-storage-day-2022/ and we got a couple (small) announcements

(Also, I might not make it to today’s call. Sorry!)

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

Andrew Hall has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

dag viggo lokoeen has joined Public "Office Hours"

Jonathan Poczatek has joined Public "Office Hours"

Linda Pham (Cloud Posse) has joined Public "Office Hours"

Andrew Vitko has joined Public "Office Hours"

Robert Jordan has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Ralf Pieper has joined Public "Office Hours"

Oscar Jara has joined Public "Office Hours"

Barak Griffis has joined Public "Office Hours"

Alexandr Vorona has joined Public "Office Hours"

Johnmary Odenigbo has joined Public "Office Hours"

Gerard Ceraso has joined Public "Office Hours"

Jan-Arve Nygård has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Oliver Schoenborn has joined Public "Office Hours"

Ben Smith (Cloud Posse) has joined Public "Office Hours"

Shaun Wang has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Ruslan Butdayev has joined Public "Office Hours"

Mazin Ahmed has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Charles Smith has joined Public "Office Hours"

Amaan Khan has joined Public "Office Hours"

Bridget Royer has joined Public "Office Hours"

Waqar Ahmed has joined Public "Office Hours"

Roy Sprague has joined Public "Office Hours"

Antarr Byrd has joined Public "Office Hours"

Disclaimer: I haven't used this, but only thought it was novel that they are essentially fighting AWS on margin for their NAT Gateways. https://www.cohesive.net/vns3/cloud-nat/

Links from today's call:
https://github.blog/2022-07-27-planning-next-to-your-code-github-projects-is-now-generally-available/
https://github.blog/changelog/2022-08-09-github-actions-self-hosted-runners-now-support-apple-m1-hardware
https://bridgecrew.io/blog/checkov-enables-ci-cd-security-with-new-supply-chain-security-policies/
https://saveslack.com/
https://home.robusta.dev/blog/stop-using-cpu-limits/
https://www.hashicorp.com/blog/cdk-for-terraform-now-generally-available
https://aws.amazon.com/about-aws/whats-new/2022/08/amazon-ebs-crash-consistent-snapshots-subset-ebs-volumes-attached-amazon-ec2-instance/

hello

hey is anyone around?

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Robert Jordan has joined Public "Office Hours"

Marc Tamsky has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

Roy Sprague has joined Public "Office Hours"

Ralf Pieper has joined Public "Office Hours"

Linda Pham (Cloud Posse) has joined Public "Office Hours"

Marc Tamsky has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

venkata mutyala has joined Public "Office Hours"

Jonathan Poczatek has joined Public "Office Hours"

Thomas Poetke has joined Public "Office Hours"

Andrew Vitko has joined Public "Office Hours"

Ronnie Coleman has joined Public "Office Hours"

dag viggo lokoeen has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Johnmary Odenigbo has joined Public "Office Hours"

Jesus Martinez has joined Public "Office Hours"

Marc Tamsky has joined Public "Office Hours"

Oskar Maria Grande has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

emem u has joined Public "Office Hours"

Matthew Pickens has joined Public "Office Hours"

Allen Lyons has joined Public "Office Hours"

Steven Kalt has joined Public "Office Hours"

Adam Buggia has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

I recently heard there are cheaper alternatives to AWS Support provided by their third party partners. I'm unclear on which partners offer this but does anyone here use an alternative to AWS Support? If so, who do you pay and is it just as "good"?

Sam Caneer has joined Public "Office Hours"

Adedapo Ajuwon has joined Public "Office Hours"

Linda Pham (Cloud Posse) has joined Public "Office Hours"

emem u has joined Public "Office Hours"

Links from today's call:
https://github.com/cloudposse/terraform-aws-ecr-public
https://github.com/cloudposse/terraform-aws-lakeformation
https://github.com/cloudposse/terraform-aws-glue
https://github.com/cloudposse/terraform-aws-athena
https://github.com/cloudposse/terraform-aws-kinesis-stream
https://github.blog/changelog/2022-08-10-github-pages-builds-with-github-actions-ga/
https://www.theregister.com/2022/08/12/atlassian_cloud_6999_2023/
https://github.com/jckuester/terradozer
https://github.com/DontShaveTheYak/cf2tf
https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-server-access-logging.html

A dev asked me today whether he should store the rds-ca root cert in his Lambda's repo (feels icky, would need a code change to update), add it at build time (redeploy to update, not loads better), or get it at run time (potentially adding an HTTP roundtrip to the front of every exec). Wondered if we could store in SSM and inject straight into the context as you can with ECS tasks, but couldn't see how. Wondered whether Lambda's temp storage persisted predictably enough that you could 'memoize'/'cache' the runtime get. Ran out of ideas. Posted in #office-hours. Any wisdom?

@here office hours is starting in 30 minutes! Remember to post your questions here.

Q: Is there a deep-dive video covering the Cloudposse Way? Things like how contexts work, incl. the fixtures files, as well as overall phlosophy, etc.?

Also, I'm building out TF for Aurora and would appreciate any input on the process.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Robert Jordan has joined Public "Office Hours"

Adnan M. has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Marc Tamsky has joined Public "Office Hours"

Vlad Ionescu has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Allan Swanepoel has joined Public "Office Hours"

venkata mutyala has joined Public "Office Hours"

Vincent Werner has joined Public "Office Hours"

Linda Pham (Cloud Posse) has joined Public "Office Hours"

Luis Masaya has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

Ralf Pieper has joined Public "Office Hours"

Bridget Royer has joined Public "Office Hours"

Roy Sprague has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Marcos Soutullo has joined Public "Office Hours"

venkata mutyala has joined Public "Office Hours"

Ralf Pieper has joined Public "Office Hours"

Oliver Schoenborn has joined Public "Office Hours"

Matthew Pickens has joined Public "Office Hours"

Alexandr Vorona has joined Public "Office Hours"

Igor M has joined Public "Office Hours"

Andrew Vitko has joined Public "Office Hours"

Adedapo Ajuwon has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Ori Yhezkel Mualem has joined Public "Office Hours"

Q: Wondering if anyone knows of operational tools (other than k8s) that manage (config-as-code) and/or sync Consul's service resolver configs:
• https://www.consul.io/docs/connect/config-entries/service-resolver

dag viggo lokoeen has joined Public "Office Hours"

Brian Choate has joined Public "Office Hours"

PePe Amengual has joined Public "Office Hours"

Sherif Abdel-Naby has joined Public "Office Hours"

Brian Choate has joined Public "Office Hours"

Mazin Ahmed has joined Public "Office Hours"

Sherif Abdel-Naby has joined Public "Office Hours"

Vincent Werner has joined Public "Office Hours"

Antonio Rodriguez has joined Public "Office Hours"

PePe Amengual has joined Public "Office Hours"

Ozzy has joined Public "Office Hours"

Antarr Byrd has joined Public "Office Hours"

Sherif Abdel-Naby has joined Public "Office Hours"

Links from today's call:
https://github.com/cloudposse/terraform-aws-dms/tree/main/examples/complete
https://github.blog/changelog/2022-08-23-github-actions-enhancements-to-openid-connect-support-to-enable-secure-cloud-deployments-at-scale
https://github.com/multycloud/terraform-provider-multy
https://docs.multy.dev/vm_sizes
https://github.com/apache/libcloud
https://registry.terraform.io/providers/devops-rob/terracurl/latest/docs/resources/request
https://github.com/SvenHamers/terraform-provider-oauth
https://doordash.engineering/2022/08/09/how-to-handle-kubernetes-health-checks/
https://aws.amazon.com/about-aws/whats-new/2022/08/aws-support-launches-managing-cases-slack/
https://github.com/apache/libcloud
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Replication.html
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.upgrade.html

There was a question regarding RDS / Database certificates today - here is a short example of how I run it on my dev machine with docker-compose

Q: Anyone use AWS code artifact? love it? hate it? We are considering it for pip. CI would push (already uses OIDC with circle CI), and devs would pull for docker container based environments. Are there any weird edges? At my last gig we were using nexus artifactory and nexus would not respect a package being yanked from pip. Nexus would continue to serve that yanked version (until a new version was pushed out) as the latest which was a hassle.

https://aws.amazon.com/blogs/aws/new-aws-support-app-in-slack-to-manage-support-cases/

this was discussed in the previous office hours, highlights on one of the points discussed.. looks like 100 account per slack channel

After you authorize your Slack workspace, you can add your Slack channels by choosing Add channel. You can add up to 20 channels for a single account. A single Slack channel can have up to 100 AWS accounts.

Q: I'm trying to import google_project_iam_policy that looks like this:

data "google_iam_policy" "iam_policy_data_mazin_test_roles_containeranalysis_ServiceAgent" {
    binding {
        role = "roles/containeranalysis.ServiceAgent"

        members = [
    "serviceAccount:service-x@container-analysis.iam.gserviceaccount.com"
]
    }
}

resource "google_project_iam_policy" "iam_policy_mazin_test_roles_containeranalysis_ServiceAgent" {
  project     = "mazin-test"
  policy_data = data.google_iam_policy.iam_policy_data_mazin_test_delete_roles_containeranalysis_ServiceAgent.policy_data
}

Based on documentation, it should be imported as:

$ terraform import google_project_iam_policy.my_project your-project-id

However when running this one followed by terraform plan, it shows that the role is being deleted instead. I'm not sure if it's related to importing data sources, or if there is a better way to import GCP IAM roles. Any ideas?

@here office hours is starting in 30 minutes! Remember to post your questions here.

Linda Pham (Cloud Posse) has joined Public "Office Hours"

venkata mutyala has joined Public "Office Hours"

Mike Martin has joined Public "Office Hours"

Vlad Ionescu has joined Public "Office Hours"

Andy Miguel (Cloud Posse) has joined Public "Office Hours"

Robert Jordan has joined Public "Office Hours"

Luis Masaya has joined Public "Office Hours"

Antonio Rodriguez has joined Public "Office Hours"

Allan Mohr has joined Public "Office Hours"

Andrew Vitko has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Ashwin Jacob has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

dag viggo lokoeen has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

Omer Sen has joined Public "Office Hours"

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Mike Drummond has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

Life Lofranco has joined Public "Office Hours"

SYED HUSSAIN has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Brian Pauley has joined Public "Office Hours"

Roy Sprague has joined Public "Office Hours"

Adnan M. has joined Public "Office Hours"

Ben Smith (Cloud Posse) has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

dag viggo lokoeen has joined Public "Office Hours"

Dana Carney has joined Public "Office Hours"

Mazin Ahmed has joined Public "Office Hours"

Vincent Werner has joined Public "Office Hours"

Ralf Pieper has joined Public "Office Hours"

Links from today's call:
https://stackoverflow.com/tags/cloudposse/info
https://aws.amazon.com/about-aws/whats-new/2022/08/amazon-cloudfront-origin-access-control/
https://sweetops.slack.com/archives/CB6GHNLG0/p1661965412589009
https://typefully.com/iamvlaaaaaaad/august-24-fargate-outage-N4rzykR
https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_example-tag-policies.html