#office-hours - November 2020

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Tarlan Isaev has joined Public "Office Hours"

Andrew Elkins has joined Public "Office Hours"

Haribabu Balagani has joined Public "Office Hours"

usha dav has joined Public "Office Hours"

Chris Dutton has joined Public "Office Hours"

Andy Roth has joined Public "Office Hours"

Nick James has joined Public "Office Hours"

David Lozano has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Andy Miguel has joined Public "Office Hours"

Patrick Joyce has joined Public "Office Hours"

Andrew Thompson has joined Public "Office Hours"

Sheldon Hull has joined Public "Office Hours"

Michael Londeen has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Tim Gourley has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Mikael Fridh has joined Public "Office Hours"

15139103984 has joined Public "Office Hours"

Kareem Shahin has joined Public "Office Hours"

Michael Martin has joined Public "Office Hours"

Alexis Concepcion has joined Public "Office Hours"

Alexis Concepcion has joined Public "Office Hours"

Andrey Nazarov has joined Public "Office Hours"

venkatamutyala has joined Public "Office Hours"

pepe amengual has joined Public "Office Hours"

Marcos Soutullo has joined Public "Office Hours"

Sebastian Borrajo has joined Public "Office Hours"

Jeremy (Cloud Posse) has joined Public "Office Hours"

Wesley Chiang has joined Public "Office Hours"

Fernando Castillo has joined Public "Office Hours"

Derek Davis has joined Public "Office Hours"

vicken has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

You could hijack DNS…

Matthew Zeemann has joined Public "Office Hours"

Matt Gowie has joined Public "Office Hours"

Marc T has joined Public "Office Hours"

• Start by checking out this long blog post which is the best introduction, summary, and reference cheat sheet on Kubernetes we have yet to find. Reviewing this will shave weeks off the learning curve.
• emprovisetech.blogspot.com/2018/12/kubernetes-container-orchestration-at.html
• kubernetesbyexample.com/
• learnk8s.io/blog/kubectl-productivity
• www.katacoda.com/courses/kubernetes

Super k8s noob here. Going thru this book and its been great so far: https://www.manning.com/books/kubernetes-in-action

I feel like Rich Hickey is a relevant tangent to this conversation: https://www.youtube.com/watch?v=kGlVcSMgtV4

I feel like the framework here is also a good read for managing k8s "at-scale"

Thanks Erik for hosting another great session.

👉️ New Zoom Recording from our Office Hours session on 2020-11-04 is now available.

Another topic for next time: SLA monitoring for upstream dependencies. If we need to make a claim against a SaaS provider, what kind of monitoring do you use to generate evidence?

Question – how to deal with resources/processes that change resource state intentionally outside of terraform?

Hi All, This is my first post/question. I’m just getting to know cloudposse/slack, and finding the few office-hours sessions I’ve been able to attend to be very insightful.
Thanks to all that are contributing!

We are working with terraform modules, and terragrunt live repo currently deploying to AWS accounts.

My question is about how resources that change (outside of terraform) are managed –
For example - we have a mongo cluster setup consisting of 9 ec2 instances in 3x3x3 configuration. (the usual mongos, config, and data nodes)
Creating this cluster in terraform is straight forward.
But as part of our security processes we have a lambda which listens to AMI update events and updates the cluster with new nodes in a very complicated dance of configuration updates.
In the end all the nodes (EC2 instances) are replaced and the cluster has had no downtime (perfect!) but, now the terraform state is completely out of sync with what is deployed.
How is this problem generally handled?

Another use-case could be similar AMI updates for an ECS cluster. In that case EC2 instances are not tracked via terraform but the autoscaling group/ launch configurations are.

Thanks in advance.
Best Regards,
Tim

Hi there, I am working implementing CIS Benchmark requirements in AWS. My goal is to go through the CIS_Amazon_Web_Services_Foundations_Benchmark_v1.3.0.pdf and manually connect the resources. Then I plan to automate the creation of the resources through terraform. I feel like this is a pretty standard thing, but I'm new to cloud and terraform. Has anyone done this before?

@here office hours is starting in 30 minutes! Remember to post your questions here.

Hi, have folks ever tried hardening the EKS worker node images and what would be the best way to go about it? Using one of the CIS hardened AMI images as a base then adding and configuring kube components to it initially comes to mind (unless one already exists?).

Jeremy (Cloud Posse) has joined Public "Office Hours"

charles pogi has joined Public "Office Hours"

Andy Roth has joined Public "Office Hours"

Tim Gourley has joined Public "Office Hours"

Alexis Concepcion has joined Public "Office Hours"

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

vicken has joined Public "Office Hours"

Vlad Ionescu has joined Public "Office Hours"

sivo has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Andy Miguel has joined Public "Office Hours"

Dale-Kurt Murray has joined Public "Office Hours"

15139103984 has joined Public "Office Hours"

Adam Kaplun has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Aravind k has joined Public "Office Hours"

emem umoh has joined Public "Office Hours"

Sam C has joined Public "Office Hours"

Chris Dutton has joined Public "Office Hours"

David Lozano has joined Public "Office Hours"

Marcos Soutullo has joined Public "Office Hours"

David Lundgren has joined Public "Office Hours"

Rainer Schuth has joined Public "Office Hours"

Rodrigo Quezada has joined Public "Office Hours"

Guelor Emanuel has joined Public "Office Hours"

Babajide Hassan has joined Public "Office Hours"

Aarat has joined Public "Office Hours"

SLOs and observability https://cloud.google.com/blog/products/gcp/defining-slos-for-services-with-dependencies-cre-life-lessons

Marc Tamsky has joined Public "Office Hours"

Mikael Fridh has joined Public "Office Hours"

Shoutout to Bottlerocket: https://aws.amazon.com/bottlerocket/

mallory mabe has joined Public "Office Hours"

having just done this - kube-bench and the Optimized Amazon Linux 2 AMI are the best way to go.

Kube-bench has a --benchmark parameter which can be set to EKS-1.0. This enables a paired down but accurate view of the Nodes.

Kube-bench also does not care if the nodes are EKS control plane managed or self-managed.

emem has joined Public "Office Hours"

emem umoh has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

@Erik Osterman (Cloud Posse), thoughts on comparisons between Terraform SaaS services?

@Erik Osterman (Cloud Posse) you mentioned before bridgecrew for enforcing sec policies in terraform. What advantage you get with this tool over TFC Sentinel sec policies?

Abisoye Olaomi has joined Public "Office Hours"

@Jake Lundberg (HashiCorp) has joined the channel

👉️ New Zoom Recording from our Office Hours session on 2020-11-11 is now available.

For the people who find out difficult to attend the office hours meetings and too lazy to watch an entire hour of video, it would be nice to have meeting minutes or short summaries to some of the discussion topics.

It would make for good sweet ops blog entries or a newsletter.

@here office hours is starting in 30 minutes! Remember to post your questions here.

Q: How are people capturing Cognito configuration via code? Anyone using Terraform to accomplish IaC for Cognito?

Pseudo-poll: Do you turn on “Dismiss PR approvals when new commits are pushed”? Why/why not?

Andy Miguel has joined Public "Office Hours"

Andy Roth has joined Public "Office Hours"

Victor Fondevilla has joined Public "Office Hours"

Oliver Schoenborn has joined Public "Office Hours"

Scott Rogers has joined Public "Office Hours"

Jagan Rajagopal has joined Public "Office Hours"

Blaise pabon has joined Public "Office Hours"

Christopher Picht has joined Public "Office Hours"

Mikael Fridh has joined Public "Office Hours"

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Sam C has joined Public "Office Hours"

I'm trying to create a kubernetes cluster in EKS, this time with the terraform-eks-aws module in terraform registry, and after a while the node group creation fails because

NodeCreationFailure: Instances failed to join the kubernetes cluster

I'm looking into it but was wondering if any had any suggestions, since this is a rather vanilla setup that I'm creating

Nicolás de la Torre has joined Public "Office Hours"

majan paul has joined Public "Office Hours"

Tomek Rabczak has joined Public "Office Hours"

Jeremy (Cloud Posse) has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Guelor Emanuel has joined Public "Office Hours"

Tim Gourley has joined Public "Office Hours"

Stefan Andonov has joined Public "Office Hours"

15139103984 has joined Public "Office Hours"

Kareem Shahin has joined Public "Office Hours"

Jagan Rajagopal has joined Public "Office Hours"

Arjun Venkatesh has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Tarlan Isaev has joined Public "Office Hours"

Juan Soto has joined Public "Office Hours"

Kevin Chan has joined Public "Office Hours"

Omer Sen has joined Public "Office Hours"

Abisoye Olaomi has joined Public "Office Hours"

Derek Davis has joined Public "Office Hours"

Yo! Could you guys please explain a little bit in details what a main purpose and power of Waypoint? Has anyone had a recent chance to touch its guts? Thanks for office hours 🙂

A few items about time:
chrony is the modern NTP client: https://chrony.tuxfamily.org/comparison.html
Software developers have a lot to learn about time: https://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time
If you know what you're doing with time, you can do some pretty cool stuff, like:
• Generate crypto (https://blog.cloudflare.com/secure-time/ )
• Google spanner database (or, how to have data every where in general and nowhere in particular)

👉️ New Zoom Recording from our Office Hours session on 2020-11-18 is now available.

@here office hours is starting in 30 minutes! Remember to post your questions here.

is there a best practice to follow w/ helmfiles in terms of inheritance / keeping things dry?

currently my setup is a remote monochart chart with zero assumptions made (i.e only default labels like app: {{ include "common.name" . }}). However I have a bunch of services that essentially follow the same pattern (all use traefik ingress annotations, redis pod). Using the traefik ingress annotation example, Instead of having all those annotations copied to every service’s helmfile, should I have a remote “parent” helmfile I can reference (is this even a thing?) or should I have a separate monochart with more assumptions (all the default traefik ingress annotations I want like HSTS enabled)

No worries! Thanks!

I put my question in the zoom chat; is it better here?

I'm just dipping my toe in with the https://github.com/cloudposse/terraform-aws-tfstate-backend - can't get it to create the bucket?!
Is there a repo that has this configured in as a working example?

@Matt Gowie What about an aws managed vpn service? https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html

I use a reverse proxy fargate cluster with okta in front on lb for aws es

mmmm I can't access it

👉️ New Zoom Recording from our Office Hours session on 2020-11-25 is now available.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

emem umoh has joined Public "Office Hours"

Andrew Thompson has joined Public "Office Hours"

Mikael Fridh has joined Public "Office Hours"

David Lundgren has joined Public "Office Hours"

David Napier has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Jeremy Branham has joined Public "Office Hours"

Andrew Thompson has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

Jeremy (Cloud Posse) has joined Public "Office Hours"

James Haughey has joined Public "Office Hours"

matt simonsen has joined Public "Office Hours"

will Salt has joined Public "Office Hours"

Patrick Joyce has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Abisoye Olaomi has joined Public "Office Hours"

Jason Walsh has joined Public "Office Hours"

Andrew Thompson has joined Public "Office Hours"

Brian Tai has joined Public "Office Hours"

Todd Thomas has joined Public "Office Hours"

Tim Gourley has joined Public "Office Hours"

Mahesh Yadav has joined Public "Office Hours"

15139103984 has joined Public "Office Hours"

Andrew Thompson has joined Public "Office Hours"

Matt Gowie has joined Public "Office Hours"

Hilal Jaffan has joined Public "Office Hours"

vicken has joined Public "Office Hours"

Matthew Zeemann has joined Public "Office Hours"

Baba Hassan has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Dale-Kurt Murray has joined Public "Office Hours"

Nick James has joined Public "Office Hours"

WTH?

ohhhhhhhh Amazon is up now LOL

no it is not....

Abisoye Olaomi has joined Public "Office Hours"

Robert Jackson has joined Public "Office Hours"

Emile Fugulin has joined Public "Office Hours"

Neil Gealy has joined Public "Office Hours"

Ion Munteanu has joined Public "Office Hours"

Bade has joined Public "Office Hours"

Abisoye Olaomi has joined Public "Office Hours"

Ah, missed office hours, sorry 😞 My schedule's super chaotic with re:Invent prep and related things. I hope I'll make it to the next one, but it's not looking good

is there a Google calendar I can subscribe to for Wednesday Office Hours by chance?

there is something somewhere because I get notifications

@PePe Amengual excellent, thank you