#office-hours - June 2023

Hey everyone, I’m still pretty new to terraform so if there’s time during the office hours today I’d love to gather input or feedback. I would like to add a dynamic block for logging filters of the terraform-aws-waf repo as I need logging filters. Basically this PR but with the necessary for_each . If there isn’t time or if this isn’t the right venue to do that I’ll just leave it async as a PR.

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) has joined Public "Office Hours"

Michael Pursifull has joined Public "Office Hours"

Austin Carter has joined Public "Office Hours"

Taylor Turner has joined Public "Office Hours"

Vlad Ionescu has joined Public "Office Hours"

Michael Jenkins has joined Public "Office Hours"

Jonathan Eunice has joined Public "Office Hours"

Nenna Salinas has joined Public "Office Hours"

Dan Hamilton has joined Public "Office Hours"

David Hawthorne has joined Public "Office Hours"

Christopher Pieper has joined Public "Office Hours"

Eric Berg has joined Public "Office Hours"

Andy Wortman has joined Public "Office Hours"

Arthur Kepler has joined Public "Office Hours"

Jim Park has joined Public "Office Hours"

Matt Calhoun has joined Public "Office Hours"

Adedapo Ajuwon has joined Public "Office Hours"

nishant kasture has joined Public "Office Hours"

venkata mutyala has joined Public "Office Hours"

Q: Is your Terraform CI/TACOS of choice still Spacelift as written here in 2021: https://cloudposse.com/faqs/why-do-you-recommend-spacelift/
?
• Has Atlantis caught up on features?
• And have you used or know of use of the fully self-hosted option (not only the self-hosted agents)
On my list to look into so far:
1. Out of scope as no self-hosted:
a. env0
b. Scalr
c. TFC
2. Self-hosted:
a. DIY: (jenkins, github actions, …)
b. Atlantis: Free&Open; No vendor support.
c. Spacelift: Paid. Not open. Claims to have fully self-hosted option. Recommended by CloudPosse.
d. Terraform Enterprise: Potentially cost-prohibitive (I have 1000s of plans/day for drift detection, and 100s of runs/day); Feedback I’ve seen is that it’s behind it’s competitors.
e. Terrateam: looks good from their website, but not clear on if many users have adopted it, how well it’s maintained and if it will survive.
f. CrossPlane: Beyond a TACOS, but worth considering?

Sean Roberts has joined Public "Office Hours"

And news worthy for those concerned with supply-chain security and compliance:
https://aws.amazon.com/blogs/containers/announcing-container-image-signing-with-aws-signer-and-amazon-eks/

Though I’m sad they chose Notary, not cosign :(

Joe Perez has joined Public "Office Hours"

Isa Aguilar has joined Public "Office Hours"

Hao Wang has joined Public "Office Hours"

Amer Zec has joined Public "Office Hours"

Links from today's office hours:

https://www.eff.org/deeplinks/2023/06/our-right-challenge-junk-patents-under-threat
https://github.com/garden-io/garden-aws-quickstart
https://garden.io/blog/aws-security-issue
https://www.infoq.com/news/2023/06/aws-documentation-github/
https://finance.yahoo.com/news/aws-announces-general-availability-amazon-200700363.html
https://trufflesecurity.com/blog/running-trufflehog-in-a-github-action/
https://youtu.be/tCfb9Wizq9Q?t=252
https://www.reddit.com/r/Terraform/comments/13vw5m7/comment/jmo8ef6/
https://aws.amazon.com/about-aws/whats-new/2023/06/live-tail-amazon-cloudwatch-logs/
https://aws.amazon.com/about-aws/whats-new/2023/06/aws-container-image-signing/
https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-ecr-registry-k8s-io-upstream-pull-through-cache-repositories/
https://aws.amazon.com/blogs/compute/ruby-3-2-runtime-now-available-in-aws-lambda/
https://www.snowflake.com/guides/using-security-data-lake-security-analytics
https://en.wikipedia.org/wiki/Google_Sidewiki
https://opensearch.org/docs/2.8/security-analytics/index/
https://aws.amazon.com/about-aws/whats-new/2023/03/amazon-opensearch-service-security-analytics/
https://github.com/github/roadmap/issues/94#issuecomment-1581086839
https://github.com/github/roadmap/issues/119#issuecomment-1581084432
https://aws.amazon.com/ecr/pricing/
https://twitter.com/matthieunapoli/status/1666199032597733380
https://github.com/TylerBrock/saw
https://aws.amazon.com/blogs/containers/announcing-container-image-signing-with-aws-signer-and-amazon-eks/
https://github.com/aws-samples/k8s-notary-admission
https://reinforce.awsevents.com/
https://pwittrock.github.io/docs/concepts/storage/volumes/#gitrepo

@here office hours is starting in 30 minutes! Remember to post your questions here.

Links from today's office hours:

https://bitfieldconsulting.com/blog/night-of-the-runbooks
https://developer.1password.com/docs/cli/shell-plugins/terraform/
https://github.blog/changelog/2023-06-13-github-actions-you-can-now-disable-repo-level-self-hosted-runners-in-an-enterprise-and-organization/
https://www.pulumi.com/blog/converting-full-terraform-programs-to-pulumi/
https://www.reddit.com/r/kubernetes/top/?t=month
https://marketplace.visualstudio.com/items?itemName=oferkafry.easy-terraform-commands
https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-inspector-code-scans-aws-lambda-function/
https://aws.amazon.com/about-aws/whats-new/2023/06/third-party-risk-assessments-csv-exports-aws-audit-manager/
https://aws.amazon.com/about-aws/whats-new/2023/06/aws-security-hub-automation-rules/
https://aws.amazon.com/about-aws/whats-new/2023/06/aws-iam-identity-center-automated-user-provisioning-google-workspace/
https://aws.amazon.com/about-aws/whats-new/2023/06/amazon-verified-permissions-generally-available/
https://aws.amazon.com/about-aws/whats-new/2023/06/software-bill-materials-export-capability-amazon-inspector/
https://aws.amazon.com/about-aws/whats-new/2023/06/aws-config-recording-exclusions-resource-type/
https://www.taccoform.com/posts/tfg_p5/
https://docs.aws.amazon.com/sdkref/latest/guide/standardized-credentials.html#credentialProviderChain
https://xkcd.com/927/
https://docs.aws.amazon.com/singlesignon/latest/userguide/scim-profile-saml.html
https://github.com/benkehoe/aws-sso-util

Curious about introducing terraform changes in a more temporal spaced way (propagating very slow through some initial stages), meanwhile other changes can be applied more rapidly in those environments.

@here office hours is starting in 30 minutes! Remember to post your questions here.

Hi folks, I have a question 🙂 Lets say im building a libary of terraform modules and publishing them to a private registry e.g. citizen. I have an internal development portal that is effectively through pipelines calling these individual modules to stand up infra resources. What would be better, establishing a means of downloading the module from the private registry? Calling the module in a .tf file within the examples directory? (But then how do i dynamically control the version o the module?)

Any experience with Galera Cluster (https://galeracluster.com/ or https://mariadb.com/kb/en/galera-cluster/)? If glossies to be believed, mutli-writer multi-master clustering for MySQL or MariaDB.

Not 100% sure if it was Galera (some time ago), but did use the multi-master setup

With success? The “did use” suggests “not using any longer.”

I moved to a different company. But I did the implementation and used it, and we were quite happy with that.

One thing we made sure that specific tables are only written/updated on a single master, so we basically sharded the tables across masters

Read is perfect across all. The setup was also used so that one master could act as a failover for another master (using a basic tcp loadbalancer doing the failover)

The basic system was that 1 master did the massive ingest of raw data, and the further processing/enhancing/summarizing was done on a second master. Further operations were done one a 3rd master.

To use fully multi-master, hitting the same tables at each master. a) they become eventually consistent, so there is a small delay b) your application and db models needs to be closely looked at

Links from today's office hours:

https://github.com/shayonj/pg_easy_replicate
https://aws.amazon.com/blogs/compute/secure-connectivity-from-public-to-private-introducing-ec2-instance-connect-endpoint-june-13-2023/
https://github.com/aws-controllers-k8s
https://github.blog/changelog/2023-06-21-github-hosted-larger-runners-for-actions-are-generally-available/
https://github.com/asannou/tfmermaid-action
https://github.com/promptops/cli
https://aws.amazon.com/about-aws/whats-new/2023/06/aws-control-tower-account-integration-security-hub/
https://ordina-jworks.github.io/cloud/2023/06/05/back-to-terraform.html
https://www.theverge.com/2023/6/16/23763340/google-domains-sunset-sell-squarespace
https://github.com/aidansteele/rdsconn
https://www.systeminit.com
https://twitter.com/adamhjk
https://aws.amazon.com/blogs/aws/new-aws-control-tower-account-factory-for-terraform/
https://github.com/aws-ia/terraform-aws-control_tower_account_factory/issues
https://twitter.com/iamvlaaaaaaad/status/1671540600976592897
https://aws.amazon.com/route53/domain-registration-agreement/
https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/find-your-registrar.html
https://neon.tech/
https://jimmyb.ninja/post/1673999840

This is cool. I missed this in their release notes for 1.5.0 the other day:

https://www.reddit.com/r/devops/comments/14gfz73/terraform_import_block_allows_to_generate_code/


Haven’t tried it yet myself but if you need to import a bunch of resources that were created in the GUI this should speed up your workflow.

Was going through some recent release notes and noticed these features for the recently released Hashicorp Vault 1.14.0

- Environment Variables through Vault Agent: Introducing a new process-supervisor mode for Vault Agent which allows injecting secrets as environment variables into a child process using a new env_template configuration stanza. The process-supervisor configuration can be generated with a new vault agent generate-config helper tool. [GH-20530]
- Vault PKI ACME Server: Support for the ACME certificate lifecycle management protocol has been added to the Vault PKI Plugin. This allows standard ACME clients, such as the EFF's certbot and the CNCF's k8s cert-manager, to request certificates from a Vault server with no knowledge of Vault APIs or authentication mechanisms. For public-facing Vault instances, we recommend requiring External Account Bindings (EAB) to limit the ability to request certificates to only authenticated clients. [GH-20752]

ref: https://github.com/hashicorp/vault/releases/tag/v1.14.0

@here office hours is starting in 30 minutes! Remember to post your questions here.

Hi, I'm Chris, an individual who's new to DevOps. (In other words, not a prospective client at the moment, just a new engineer.)

I came across you modules on the TF registry and was curious about what you consider a professional workflow for Terraform deployments. The ultimate goal seems to be semi automated code review and automated test deployments to an isolated account/network.

But getting to that last step take a ton of effort. What's do you think is the sweet spot?

Links from today's office hours:

https://changie.dev/
https://masterpoint.io/updates/passing-on-crossplane/
https://www.linkedin.com/feed/update/urn:li:activity:7077737001386455040/?utm_source=share&utm_medium=member_desktop
https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/
https://www.bleepingcomputer.com/news/security/lastpass-users-furious-after-being-locked-out-due-to-mfa-resets/
https://www.dispatch.do/
https://github.blog/changelog/2023-06-27-github-actions-update-on-oidc-integration-with-aws/
https://medium.com/@DiggerHQ/you-can-now-import-your-existing-infrastructure-into-terraform-now-what-7d7bfe4d9334
https://github.com/spilliams/terrascope
https://openai.com/research/scaling-kubernetes-to-7500-nodes
https://github.com/aws/aws-application-networking-k8s
https://www.beeper.com/
https://meetfranz.com/
https://github.com/wazuh/wazuh
https://wazuh.com/
https://www.systeminit.com/
https://twitter.com/iamvlaaaaaaad/status/1671540600976592897