#terraform - February 2024

Hi! Could anyone have a look at the PR? It’s hanging about for some time
CC @Andriy Knysh (Cloud Posse)
https://github.com/cloudposse/terraform-aws-config/pull/83

Terraform Stacks preview: https://www.youtube.com/watch?v=8p_o-Me8VuA

Any chance cloud posse would be interested in maintaining a fork of terraform docs to unblock prs?

Could use some help here if anybody has a moment to take a look, thanks
https://sweetops.slack.com/archives/CDYGZCLDQ/p1707013439249969

Hi all, wonder if i can get some feeback/advice of what the best approach for using terraforma and ansible together. Is there an ansible provider that facilitates the configuration of my servers? Thanks in advance!

AWS/Terraform question: Given multiple elastic beanstalk environments that all utilize a shared ALB, does anybody know how to add custom ALB rules to the listener so that each rule maps each environment's custom DNS to its designated beanstalk target group?

What is happening right now is beanstalk is creating a listener rule that uses the beanstalk DNS app-name.random-characters.us-east-1.elasticbeanstalk.com but I need this rule to have the proper host headers for the app... like *.app-domain.com

I tried this setting but it doesn't appear to have any effect on the listener at all...

{
      namespace: "aws:elbv2:listenerrule:${each.value}",
      name: "HostHeaders",
      value: "${each.value}.${aws_route53_zone.private.name}",
    },

Does anyone know why the homebrew formula for terraform says not to bump to v1.6 because of the license change? Curious why brew is affected

Few questions about the MSK module:
1. You can work with SASL Scram or SASL Iam with MSK that has Kafka version of 2.5.1 ? how can I know if version is too old to support some kind of encryption ?
2. When you create a new MSK cluster, you are only enable the unauthenticated option (via variable client_allow_unauthenticated) - It should be good. But what happen when you changing the client_authentication methods and enable SASL Scram and/or SASL Iam ?
The terraform module will handle it right ?

Im asking because in our own MSK module we're also using the Kafka provider to create the kafka topics within the MSK cluster.
I think that because they both combine to one module there's some mismatch between the MSK and aws provider and the kafka provider. some weird behavior when trying to update existing cluster auth method from unauthenticated.
even when the kafka provider has bootstrap_servers set we are getting this error when trying to apply some changes:

No bootstrap_servers provided 

v1.7.3
1.7.3 (February 7, 2024)
BUG FIXES:

terraform test: Fix crash when dynamic-typed attributes are not assigned values in mocks. (#34610)
provisioners/file: Fix panic when source is null. (<a href="https://github.com/hashicorp/terraform/pull/34621" data-hovercard-type="pull_request"...

Prismacloud is killing the free checkov vscode extension by paywalling the api key.
https://github.com/bridgecrewio/checkov-vscode/issues/141

I have an interesting challenge. There’s a very large terraform codebase (thousands resources) and currently the team relies on cloud resource names (e.g. s3 bucket name) to find those resources in terraform code. Literally, searching bucket name in codebase.

We want to introduce terraform-null-label for consistency. However, it will break the current way of finding resources in the tf codebase.

We tried to use yor but it adds too much complexity and also introduces an extra step - to find a resource in the code, you need first lookup tags (e.g. s3 bucket tags) and then using those meta-tag values to search in the codebase. Ideally, we want to avoid it.

There’s also no access to state. Only tf codebase.

Question: Has anyone tried to pre-generate null-label resource names and add them to terraform files as comments? Any other ideas how to map cloud resource names to terraform files in a large codebase?

hi all 👋,
new to the group...

I'm facing an issue in creating Datadog monitor.
getting this error, saying the module datadog_monitors is not found.... all of a sudden starting today mng

╷
│ Error: Module not found
│ 
│ Module "datadog_monitors" (from <http://main.tf:14|main.tf:14>) cannot be found in the module
│ registry at <http://registry.terraform.io|registry.terraform.io>.
╵

it used to work with below version:

module "datadog_monitors" {
  source  = "cloudposse/monitor/datadog"
  version = "1.3.0"

started seeing the issue today mng..

We just rolled out a big update to our atmos docs. As part of that we published a write up of the limitations we encountered with standalone terraform that led to #atmos, or have seen customers encounter. We break it into 10 stages, with the 10th stage being terraform bankruptcy.

https://atmos.tools/reference/terraform-limitations

What do you do when you have one resource/module that you want to deploy in multiple accounts and it needs to be the same in all accounts?
Is there any other way other than copy/pasting the resource/module x times with different providers?

What I'd like to do is something like this

module "example" {
  source = "source"
  providers {
    aws = aws.one
    aws = aws.two
    aws = aws.three
  }
  some_var = aws.alias
  # rest of config is the same
}

v1.8.0-alpha20240214
1.8.0-alpha20240214 (February 14, 2024)
UPGRADE NOTES:

The first plan after upgrading may show resource updates with no apparent changes if -refresh-only or -refresh=false is used. The fix introduced for <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2098393853" data-permission-text="Title is private" data-url="https://github.com/hashicorp/terraform/issues/34567" data-hovercard-type="pull_request"...

Hi - Is there a way to merge multiple policy statements using terraform? I have a bunch of json files for each service. I want to create a iam policy based on user input of services.

Hello! I'm working in your api gw module. I wanted to know how i'm supposed to get custom access logs working?

I have the following variables related to logging, but as the screenshot shows, custom access logging is not turned on.

  xray_tracing_enabled = true #X-Ray tracing
  metrics_enabled = true #Detailed metrics
  logging_level = "INFO"
  #log_group_arn = "arn:aws:logs:us-east-1:829505554415:log-group:blue-parakeet"
  access_log_format = <redacted for simplicity>

i've already run the account-settings module once per region

v1.8.0-alpha20240216
1.8.0-alpha20240216 (February 16, 2024)
UPGRADE NOTES:

The first plan after upgrading may show resource updates with no apparent changes if -refresh-only or -refresh=false is used. The fix introduced for <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2098393853" data-permission-text="Title is private" data-url="https://github.com/hashicorp/terraform/issues/34567" data-hovercard-type="pull_request"...

@Everyone
Hi all friends
can anyone help me out to refer a terraform modules where root module called child module as a source and create required resources from locals like
locals{
}
module "rds" {
for_each = { for k, v in local.rds : k => v if try(v.create, true) }
source = ./modules/rds
}

Hi folks!
I am trying to create a “composite” DataDog monitor (https://registry.terraform.io/providers/DataDog/datadog/latest/docs/guides/monitors#composite-monitors).

We have some alarms defined in .yaml, like these: https://github.com/cloudposse/terraform-datadog-platform/blob/main/catalog/monitors/k8s.yaml.

I couldn’t find an example composite alarm in the repo, and am struggling to piece together a query that DataDog validates successfully.
Using some existing monitors here for a simple composite alarm example, should something like this work?

k8s-high-cpu-usage:
...
k8s-high-disk-usage:
...

k8s-cpu-disk-composite:
  name: "(k8s) High CPU and High Disk Usage Detected"
  type: composite
  query: |
    datadog_monitor.k8s-high-cpu-usage.id || datadog_monitor.k8s-high-disk-usage.id 
...

I tested a query of 123456789 || 987654321 , and that works OK. So it just seems to be a problem of grabbing those IDs.
Also tried k8s-high-cpu-usage.id || k8s-high-disk-usage.id, but that also had validation issues.

v1.7.4
1.7.4 (February 21, 2024)
BUG FIXES:

terraform test: Fix automatic loading of variable files within the test directory on windows platforms. (#34666)
plan renderer: Very large numbers (> 2^63) will no longer be truncated in the human-readable plan. (<a href="https://github.com/hashicorp/terraform/pull/34702"...

Hi Guys, I'm having an issue I can't seem to figure out. I'm working on a few terraform modules for our application and one of them is a AWS Transfer Family module. Everything so far appears to be working, however, I can't figure out why it's not picking up the EIP allocation ids for the endpoint details, everything else in the endpoint details works.

The specific area I'm having trouble with is below. Everything else works, like the similar security_group_ids line above it) and I've verified the EIP's are created using terraform state show. The weird thing is, if I change "address_allocation_ids" in the lookup to anything else, like "foo", it picks up the EIPs the module created and works.

  dynamic "endpoint_details" {
    for_each = var.transfer_server_type == "VPC" || var.transfer_server_type == "VPC_ENDPOINT" ? ["enabled"] : []
    content {
      vpc_id                 = lookup(var.endpoint_details, "vpc_id", null)
      vpc_endpoint_id        = lookup(var.endpoint_details, "vpc_endpoint_id", null)
      subnet_ids             = lookup(var.endpoint_details, "subnet_ids", null)
      security_group_ids     = lookup(var.endpoint_details, "security_group_ids", aws_security_group.this[*].id)
      address_allocation_ids = lookup(var.endpoint_details, "address_allocation_ids", aws_eip.this[*].allocation_id)
    }
  }

Hi, I have a question regarding Permission Sets on SSO. Currently I'm working with permission sets without problems. I've tried to add a new one that has a customer_managed_policy_attachments , but I'm not able to make it work.

I followed up the documentation from the components but I always get this error:

Error: waiting for SSO Permission Set (arn:aws:sso:::permissionSet/ssoins-7223c730732a9a98/ps-ffa7195776e1bd0d) provision: unexpected state 'FAILED', wanted target 'SUCCEEDED'. last error: Received a 404 status error: Not supported policy arn:aws:iam::999999999999:policy/RedshiftManagement.

What I saw, and I don't know if this is the issue is that the Customer Managed Policy RedshiftManagement is created under the root account 111111111111, but it needs to be attached to the permission Set at the Account 999999999999 .
Here is the complete code of the Permission Set:

locals {
  red_shift_access_permission_set = [{
    name             = "RedshiftAccess",
    description      = "Allow access to Redshift",
    relay_state      = "",
    session_duration = "",
    tags             = {},
    inline_policy    = data.aws_iam_policy_document.SqlWorkbench.json,
    policy_attachments = [
      "arn:${local.aws_partition}:iam::aws:policy/AmazonRedshiftDataFullAccess",
      "arn:${local.aws_partition}:iam::aws:policy/AmazonRedshiftQueryEditorV2FullAccess",
      "arn:${local.aws_partition}:iam::aws:policy/AmazonRedshiftFullAccess"
    ]
    customer_managed_policy_attachments = [
      {
        name = aws_iam_policy.RedshiftManagement.name
        path = aws_iam_policy.RedshiftManagement.path
      }
    ]
  }]
}

resource "aws_iam_policy" "RedshiftManagement" {
  name   = "RedshiftManagement"
  path   = "/"
#  policy = aws_iam_policy_document.RedshiftManagement.json
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect   = "Allow",
        Action   = "s3:GetObject",
        Resource = "*",
      },
    ],
  })
}

data "aws_iam_policy_document" "SqlWorkbench" {
  statement {
    sid       = "SqlWorkbenchAccess"
    effect    = "Allow"
    actions   = ["sqlworkbench:*"]
    resources = ["*"]
  }
  statement {
    sid    = "s3Actions"
    effect = "Allow"
    actions = [
      "s3:PutObject",
      "s3:Get*",
      "s3:List*"
    ]
    resources = [
      "arn:aws:s3:::aaaaaa-redshift",
      "arn:aws:s3:::aaaaaa-redshift/*"
    ]
  }
}

Here RedshiftManagement has a simple action to test the policy attachment.

v1.8.0-alpha20240228
1.8.0-alpha20240228 (February 28, 2024)
UPGRADE NOTES:

The first plan after upgrading may show resource updates with no apparent changes if -refresh-only or -refresh=false is used. The fix introduced for <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2098393853" data-permission-text="Title is private" data-url="https://github.com/hashicorp/terraform/issues/34567" data-hovercard-type="pull_request"...

This is fascinating to me: Someone has written their own full-blown Terraform framework for themselves + a complete Stacks alternative.
https://github.com/DavidGamba/dgtools/tree/master/bt#stacks-a-different-take
https://github.com/DavidGamba/dgtools/tree/master/bt

Is there a way to merge aws_secretsmanager, values, I’m trying to create a secret that I can add to. My main problem is the first run when there is no secret yet.

i work with the api-gateway module a lot. i'm curious how it's recommended to manage the lambda permissions that go along with resources configured inside the api-gw. Say, if a resource used a lambda function for it's integration request, that lambda would need a policy statement for invoke permissions for that api gw. i see there isn't lambda permission code in the module. i'm working on a separate module to add permissions after the gateway is deployed. i wanted to know if the cloudposse team had any discusssions on navigating this when creating the code for the module.