#office-hours - April 2023

Thing related to the discussion we had last week: https://github.com/toricls/aws-fargate-with-step-functions

Is there any video library link where we can see all office hours

If yes pls share link

Thanks in advance

I’m setting up an Elastic stack (Elasticsearch, Kibana) which will mainly be ingesting time-based data (via Filebeat/Metricbeat in EKS clusters); this new stack will replace an older, legacy Elastic stack (version 7.9). This is a self-hosted stack deployed as a series of AWS ASGs.

The legacy stack is only configured to use the hot tier. I’d like to set up an ILM policy for the new stack to leverage hot/warm/cold tiers.

Few questions:
1. Do you have recommendations on an example ILM policy that we could start with?
2. What instance types or other resources would you recommend for each data tier?

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Vlad Ionescu has joined Public "Office Hours"

Taylor Turner has joined Public "Office Hours"

Linda Pham (Cloud Posse) has joined Public "Office Hours"

Marc Slayton has joined Public "Office Hours"

Amer Zec has joined Public "Office Hours"

Andy Wortman has joined Public "Office Hours"

Jeremy / Nuru (Cloud Posse) has joined Public "Office Hours"

Paul Marcelin has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Jeremy Bouse has joined Public "Office Hours"

tyler has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

Fireflies.ai Notetaker has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

dave lundgren has joined Public "Office Hours"

dag viggo lokoeen has joined Public "Office Hours"

Roy Sprague has joined Public "Office Hours"

Andrew Thompson has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Jonathan Eunice has joined Public "Office Hours"

David Hawthorne has joined Public "Office Hours"

Risikat Yewande Raji has joined Public "Office Hours"

Alejandro Calbazana has joined Public "Office Hours"

Kris Musard has joined Public "Office Hours"

Alejandro Calbazana has joined Public "Office Hours"

Jeremy White has joined Public "Office Hours"

Matt Gowie has joined Public "Office Hours"

Oskar Maria Grande has joined Public "Office Hours"

Michael Vasilenko has joined Public "Office Hours"

Radha Ramadoss has joined Public "Office Hours"

Anup Dubey has joined Public "Office Hours"

Michael Vasilenko has joined Public "Office Hours"

Radha Ramadoss has joined Public "Office Hours"

Links from today's office hours:
https://spacelift.io/blog/introducing-spacelift-self-hosted
https://aws.amazon.com/about-aws/whats-new/2023/04/aws-service-catalog-terraform-open-source/
https://github.com/aws-samples/service-catalog-engine-for-terraform-os
https://github.blog/2023-03-28-introducing-self-service-sboms/
https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/network/dependencies
https://aws.amazon.com/blogs/aws/simplify-service-to-service-connectivity-security-and-monitoring-with-amazon-vpc-lattice-now-generally-available/
https://github.com/hashicorp/terraform-provider-aws/issues/30380
https://github.com/AlexNabokikh/tfsort?
https://github.com/sylwit/terraform-cleaner
https://github.com/SpotOnInc/renovate-config
https://github.com/alfasoftware/astra
github/cloudposse/packages
Promptops.com
https://github.com/jerryjliu/llama_index
https://github.com/marketplace/actions/gitops-automatic-versioning

did you guys discussed this before?

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Taylor Turner has joined Public "Office Hours"

Michael Pursifull has joined Public "Office Hours"

Nenna Salinas has joined Public "Office Hours"

When updating EKS how do you folks do it? I tried doing it via terraform and got the error below, but when i updated it manually in the AWS Console/UI it worked fine and my terraform plan was clean without changes.

When using the cloudposse modules to manage EKS how do you folks do your EKS upgrades?


module.cluster.aws_iam_role_policy_attachment.ebs_csi: Refreshing state... [id=AmazonEKS_EBS_CSI_DriverRole-2023041217083972830000000a]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place
 <= read (data resources)

Terraform planned the following actions, but then encountered a problem:

  # module.cluster.module.kubernetes.data.aws_eks_cluster_auth.eks[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_eks_cluster_auth" "eks" {
      + id    = (known after apply)
      + name  = "cluster"
      + token = (sensitive value)
    }

  # module.cluster.module.kubernetes.data.tls_certificate.cluster[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "tls_certificate" "cluster" {
      + certificates = (known after apply)
      + id           = (known after apply)
      + url          = "<https://oidc.eks.us-west-2.amazonaws.com/id/190881CA4EA9C23E6847BDCA0F149F8F>"
    }

  # module.cluster.module.kubernetes.aws_eks_cluster.default[0] will be updated in-place
  ~ resource "aws_eks_cluster" "default" {
        id                        = "cluster"
        name                      = "cluster"
        tags                      = {
            "Attributes" = "cluster"
            "Name"       = "cluster"
        }
      ~ version                   = "1.24" -> "1.25"
        # (10 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

  # module.cluster.module.kubernetes.aws_iam_openid_connect_provider.default[0] will be updated in-place
  ~ resource "aws_iam_openid_connect_provider" "default" {
        id              = "arn:aws:iam::739737747774:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/190881CA4EA9C23E6847BDCA0F149F8F"
        tags            = {
            "Attributes" = "cluster"
            "Name"       = "cluster"
        }
      ~ thumbprint_list = [
          - "9e99a48a9960b14926bb7f3b02e22da2b0ab7280",
        ] -> (known after apply)
        # (4 unchanged attributes hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.
╷
│ Error: configmaps "aws-auth" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
│ 
│   with module.cluster.module.kubernetes.kubernetes_config_map.aws_auth_ignore_changes[0],
│   on .terraform/modules/cluster.kubernetes/auth.tf line 118, in resource "kubernetes_config_map" "aws_auth_ignore_changes":
│  118: resource "kubernetes_config_map" "aws_auth_ignore_changes" {
│ 

Alex Atkinson has joined Public "Office Hours"

Robert Jordan has joined Public "Office Hours"

venkata mutyala has joined Public "Office Hours"

David Hawthorne has joined Public "Office Hours"

Ian Bartholomew has joined Public "Office Hours"

Joe Caulfield has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

Ozzy has joined Public "Office Hours"

tyler has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Andrew Vitko has joined Public "Office Hours"

AJ Junior has joined Public "Office Hours"

John Mitchell has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Johnmary Odenigbo has joined Public "Office Hours"

Antarr Byrd has joined Public "Office Hours"

Allen Lyons has joined Public "Office Hours"

Jeremy White has joined Public "Office Hours"

Adedapo Ajuwon has joined Public "Office Hours"

Kris Musard has joined Public "Office Hours"

mtamsky has joined Public "Office Hours"

Marc Slayton has joined Public "Office Hours"

Brandon Nason has joined Public "Office Hours"

Ozzy has joined Public "Office Hours"

Ralf Pieper has joined Public "Office Hours"

Ozzy has joined Public "Office Hours"

Links from today's office hours:

https://atmos.tools/cli/configuration/#logs
https://clark.center/browse?collection=nccp&currPage=1
https://bashdb.sourceforge.net/remake/
https://www.hashicorp.com/blog/vault-secrets-operator-a-new-method-for-kubernetes-integration
https://www.hashicorp.com/blog/introducing-hermes-an-open-source-document-management-system
https://github.com/future-architect/tftarget
https://aws.amazon.com/about-aws/whats-new/2023/04/aws-app-runner-compute-configurations/
https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-kubernetes-version-1-26/
https://news.ycombinator.com/item?id=32600821
http://mynixos.com/
https://www.chezmoi.io/
https://github.com/aws-samples/eks-cluster-upgrade
https://github.com/terraform-aws-modules
https://github.com/vmware-archive/kubewatch
https://github.com/hwchase17/langchain

yes @Erik Osterman (Cloud Posse) with 12 GB and 4vCpus you can definitely run atlantis in app-runner

Compliance has been around for a while, but still I see folks thinking that they can "agreement" away the personal rights of their data subjects.
As far as I know regulations are pretty straight forward. You collect Protected Data, you're subject to the related laws, regardless of whether your product is "beta". This article from Instabug suggest otherwise with statements like: "the developer disclaims any liability for data loss, damages", and "Testing is the only purpose behind using the application and the developer disclaims any liability for data loss, damages,...".

Is there a resource, such as a guide, that someone knows about that specifically kneecaps this line of thought? As far as I'm concerned, this shouldn't need such elaboration, but people.........
Unless I'm 100% wrong and you can easily skirt data protection regulations... But if that's the case, why is compliance and infosec a billion dollar industry? So I'm pretty sure this isn't the case. 😛

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Oliver Schoenborn has joined Public "Office Hours"

Rizky Ramadhan has joined Public "Office Hours"

Nenna Salinas has joined Public "Office Hours"

Robert Jordan has joined Public "Office Hours"

Andy Wortman has joined Public "Office Hours"

Andrew Vitko has joined Public "Office Hours"

Jacob Hudson has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

Jared Richards has joined Public "Office Hours"

Opeyemi folorunsho has joined Public "Office Hours"

Michael Holt has joined Public "Office Hours"

Abdul Aziz Tetteh has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

Michael Pursifull has joined Public "Office Hours"

Diego Maia has joined Public "Office Hours"

Eduardo Wohlers has joined Public "Office Hours"

Chad Thomas has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

PePe Amengual has joined Public "Office Hours"

Jim C has joined Public "Office Hours"

Kris Musard has joined Public "Office Hours"

Abdul Aziz Tetteh has joined Public "Office Hours"

Rizky Ramadhan has joined Public "Office Hours"

Andrew Thompson has joined Public "Office Hours"

Joko Sarjono has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Paul Marcelin has joined Public "Office Hours"

Adedapo Ajuwon has joined Public "Office Hours"

Austin Blythe has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

What have you found is THE simplest way to deploy a container for a prototype application? 🙂 ECS+fargate?

Amaan Khan has joined Public "Office Hours"

Links from today's office hours:

https://registry.terraform.io/modules/cloudposse/amplify-app/aws/latest
https://github.com/kayac/ecspresso
https://github.com/cloudposse/github-action-release-label-validator
https://github.com/cloudposse/github-action-release-branch-manager
https://github.com/cloudposse/github-action-major-release-tagger
https://www.chatpdf.com/
https://techcrunch.com/2023/04/19/atlassian-brings-an-ai-assistant-to-jira-and-confluence/amp/?
https://betterprogramming.pub/iac-secrets-management-from-github-secrets-to-aws-secrets-manager-63bb4fdd5992
https://github.com/jatalocks/terracove
https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-efs-10-gibs-throughput/
https://github.com/dagger/dagger
https://sweetops.slack.com/archives/CHDR1EWNA/p1681756391429779
https://github.com/cloudposse/terraform-aws-components

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Vlad Ionescu has joined Public "Office Hours"

Kevin Moore has joined Public "Office Hours"

Nenna Salinas has joined Public "Office Hours"

Alejandro Calbazana has joined Public "Office Hours"

Robert Jordan has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Jonathan Eunice has joined Public "Office Hours"

Isaac M has joined Public "Office Hours"

Alex Atkinson has joined Public "Office Hours"

Diego Maia has joined Public "Office Hours"

Alejandro Calbazana has joined Public "Office Hours"

Michael Pursifull has joined Public "Office Hours"

John Mitchell has joined Public "Office Hours"

Kris Musard has joined Public "Office Hours"

Jared Richards has joined Public "Office Hours"

Jonathan Poczatek has joined Public "Office Hours"

PePe Amengual has joined Public "Office Hours"

Max Lobur has joined Public "Office Hours"

emem u has joined Public "Office Hours"

Charles Smith has joined Public "Office Hours"

Adedapo Ajuwon has joined Public "Office Hours"

Evan Pitstick has joined Public "Office Hours"

Charles Smith has joined Public "Office Hours"

Abdul Aziz Tetteh has joined Public "Office Hours"

Alejandro Calbazana has joined Public "Office Hours"

Jose Figueredo has joined Public "Office Hours"

04:09:33 - VPC CNI Network Policy Support
04:10:23 - Cluster Access Management
04:12:03 - IRSAv2
04:16:23 - Cluster updates
04:23:58 - Karpenter Automatic Compute

Michael Pursifull has joined Public "Office Hours"

Links from today's office hours:

https://infracopilot.io/
https://aws.amazon.com/about-aws/whats-new/2023/03/amazon-guardduty-monitors-runtime-activity-containers-eks/
https://github.com/robusta-dev/krr
https://github.blog/changelog/2023-04-20-github-actions-create-and-share-your-own-deployment-protection-rules-for-safe-and-controlled-deployments/
https://github.com/tf-libsonnet/core
https://www.thoughtworks.com/radar?utm_source=marketo&utm_medium=email&utm_campaign=techradar-vol28_2023-04&mkt_tok=MTk5LVFERS0yOTEAAAGLW_VMeFYc69h2Mrph2Fz21dF73QVy2VL6vkxJHEL95eBNlyOojdNoqXFgWaSuKRXyRNalW7SiQgvgM6JmbR6thye_lD1TYRVUBZAsXHDTDnYGlqc
https://www.reddit.com/r/Terraform/comments/12yvukb/mac_literally_just_started_blocking_terraform/
https://github.com/stefanprodan/timoni
https://atlasgo.io/blog/2023/04/21/terraform-v050
https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-ec2-ubuntu-pro-subscription-model/
https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-guardduty-aws-lambda/
https://www.youtube.com/watch?v=LGD52z0LxAA
https://github.com/klothoplatform/klotho
https://ermetic.com/solution/just-in-time/
https://github.com/bmarsh9/gapps
https://github.com/strongdm/comply
https://github.com/cloudposse/terraform-aws-amplify-app
https://www.datacenterdynamics.com/en/news/water-leak-at-paris-global-switch-data-center-causes-fire-leads-to-outages-at-google/