#build-harness - December 2019 | Slack Archive

build-harnessArchived

119 messages

Help with the Cloud Posse build-harness https://github.com/cloudposse/build-harness

December 2019

Jan 2020

erikabout 6 years ago

@Erik Osterman (Cloud Posse) has joined the channel

erikabout 6 years ago

set the channel description: Help with the Cloud Posse build-harness https://github.com/cloudposse/build-harness

Andriy Knysh (Cloud Posse)about 6 years ago

@Andriy Knysh (Cloud Posse) has joined the channel

Erik Osterman (Cloud Posse)about 6 years ago

@Igor just created this channel

Igorabout 6 years ago

@Igor has joined the channel

Erik Osterman (Cloud Posse)about 6 years ago

@Andriy Knysh (Cloud Posse) said you have some problems with the build-harness when running make readme

Erik Osterman (Cloud Posse)about 6 years ago

I'd like to help triage

Igorabout 6 years ago

Yes, thank you. The readme is generated but without terraform output. ie docs/terraform.md is empty.

Andriy Knysh (Cloud Posse)about 6 years ago

I had similar issue (variables/outputs not generated). It was an old version of gomplate in PATH

Andriy Knysh (Cloud Posse)about 6 years ago

removed it, and then make readme/deps installed the new version, which worked

Igorabout 6 years ago

I have no gomplate in PATH

Andriy Knysh (Cloud Posse)about 6 years ago

but that was on Mac

Erik Osterman (Cloud Posse)about 6 years ago

ok, so docs/terraform.md does not have anything to do with gomplate

Erik Osterman (Cloud Posse)about 6 years ago

so we can rule that out

Igorabout 6 years ago

That's terraform-docs, right?

Erik Osterman (Cloud Posse)about 6 years ago

docs/terraform.md is generated using the "terraform-docs" or more accurately a shell script that wraps it and munges the terraform code (HCL2) so that it works with terraform-docs

Erik Osterman (Cloud Posse)about 6 years ago

i suspect there's something we are using in that script that doesn't work on your ubuntu distro (perhaps missing package)

Erik Osterman (Cloud Posse)about 6 years ago

one sec

Erik Osterman (Cloud Posse)about 6 years ago

https://github.com/cloudposse/build-harness/blob/master/bin/terraform-docs.sh

Erik Osterman (Cloud Posse)about 6 years ago

this is the script we run

Erik Osterman (Cloud Posse)about 6 years ago

can you test if that script works for you?

Erik Osterman (Cloud Posse)about 6 years ago

e.g. ./build-harness/bin/terraform-docs.sh md . i think

Igorabout 6 years ago

I don't have terraform-docs in my PATH either

Igorabout 6 years ago

but it seems to be in the /vendor/ folder of the build-harness

Erik Osterman (Cloud Posse)about 6 years ago

https://github.com/cloudposse/build-harness/blob/master/Makefile#L4

Erik Osterman (Cloud Posse)about 6 years ago

the Makefile sets the search PATH to include the build-harness/vendor folder

Igorabout 6 years ago

Do I need to run make within build-harness itself?

Igorabout 6 years ago

(hides behind the tree)

Erik Osterman (Cloud Posse)about 6 years ago

not technically, you just need those things in your path

Erik Osterman (Cloud Posse)about 6 years ago

so try this:

Erik Osterman (Cloud Posse)about 6 years ago

(for debugging)
export PATH="./build-harness/vendor:$PATH"

Erik Osterman (Cloud Posse)about 6 years ago

then if you run terraform-docs it should find it

Erik Osterman (Cloud Posse)about 6 years ago

test that that works first..

Erik Osterman (Cloud Posse)about 6 years ago

then we'll proceed to debug

Jeff Youngabout 6 years ago

@Jeff Young has joined the channel

Igorabout 6 years ago(edited)

cannot open /bin/terraform-docs.awk

Erik Osterman (Cloud Posse)about 6 years ago

aha sec

Erik Osterman (Cloud Posse)about 6 years ago(edited)

set export BUILD_HARNESS_PATH=./build-harness
(again, just for debugging right now - none of this stuff needs to be set otherwise)

Igorabout 6 years ago

no output from terraform-docs.sh now

Erik Osterman (Cloud Posse)about 6 years ago

ok, run the script this way:

Erik Osterman (Cloud Posse)about 6 years ago

bash -x ./build-harness/bin/terraform-docs.sh md .

Erik Osterman (Cloud Posse)about 6 years ago

then we should see where it gets

Igorabout 6 years ago

+ which awk
+ which terraform
+ which terraform-docs
++ head -1
++ terraform version
+ [[ Terraform v0.12.18 =~ 0\.12 ]]
++ mktemp /tmp/terraform-docs-XXXXXXXXXX
+ TMP_FILE=/tmp/terraform-docs-JhZVXiFB8g
+ awk -f ./build-harness/bin/terraform-docs.awk ./main.tf ./outputs.tf ./variables.tf ./versions.tf
+ terraform-docs md /tmp/terraform-docs-JhZVXiFB8g

+ rm -f /tmp/terraform-docs-JhZVXiFB8g

Erik Osterman (Cloud Posse)about 6 years ago

Ok, so it's working, but appears then to be something wrong either with the awk

Erik Osterman (Cloud Posse)about 6 years ago

or the output from the awk

Erik Osterman (Cloud Posse)about 6 years ago

can you run awk -f ./build-harness/bin/terraform-docs.awk ./main.tf ./outputs.tf ./variables.tf ./versions.tf

Andrew Rothabout 6 years ago

@Andrew Roth has joined the channel

Erik Osterman (Cloud Posse)about 6 years ago

we also have a couple open PRs

erikabout 6 years ago

Erik Osterman (Cloud Posse)about 6 years ago

i haven't had a chance to test the work yet by @cytopia

cytopiaabout 6 years ago

@cytopia has joined the channel

Erik Osterman (Cloud Posse)about 6 years ago

do any of those issues sound like they could be related to your problem?

Erik Osterman (Cloud Posse)about 6 years ago

if so, we can test that

Igorabout 6 years ago

I messed something up in my environment and now curl to get gomplate fails for me 🤦‍♂️

Igorabout 6 years ago

on make readme/deps

Erik Osterman (Cloud Posse)about 6 years ago

what's the error?

Igorabout 6 years ago

curl: (23) Failed writing body (0 != 16360)

Erik Osterman (Cloud Posse)about 6 years ago

oh, that just means the gomplate binary is not writable

Erik Osterman (Cloud Posse)about 6 years ago

probably b/c sudo

Erik Osterman (Cloud Posse)about 6 years ago

check fs permissions

Igorabout 6 years ago

all good now

Igorabout 6 years ago

trying the awk cmd, one sec

Igorabout 6 years ago

no output

tamskyabout 6 years ago

@tamsky has joined the channel

Camilo Bravoabout 6 years ago

@Camilo Bravo has joined the channel

Dave Yuabout 6 years ago

@Dave Yu has joined the channel

oscarabout 6 years ago

@oscar has joined the channel

Erik Osterman (Cloud Posse)about 6 years ago

<https://raw.githubusercontent.com/cloudposse/build-harness/b65f55ec83c4c63d6aff1dfa2201547ffc676262/bin/terraform-docs.awk>

Erik Osterman (Cloud Posse)about 6 years ago

can you download that file

Erik Osterman (Cloud Posse)about 6 years ago

then replace the awk script with that one

Erik Osterman (Cloud Posse)about 6 years ago

re-run see if it works?

Erik Osterman (Cloud Posse)about 6 years ago

if so, I'll merge this PR

Erik Osterman (Cloud Posse)about 6 years ago

(also, i assume you can read all the *.tf files? no perms issues, right?)

Erik Osterman (Cloud Posse)about 6 years ago

btw, @Igor which repo?

Igorabout 6 years ago

terraform-aws-ecs-container-definition

Igorabout 6 years ago

What should the expected output be for terraform-docs.awk?

Igorabout 6 years ago

I get nothing with the version above as well

Erik Osterman (Cloud Posse)about 6 years ago

the terraform variables and outputs formatted in a an old HCLv1 format

Erik Osterman (Cloud Posse)about 6 years ago

that can be read by terraform-docs

Erik Osterman (Cloud Posse)about 6 years ago

Here's the issue: https://github.com/segmentio/terraform-docs/issues/62

Erik Osterman (Cloud Posse)about 6 years ago

basically we wrote an awk script to munge the HCL2 code so it could be parsed by terraform-docs (sufficient for automatic documentation)

Igorabout 6 years ago

If I run terraform-docs directly

Igorabout 6 years ago

I get 2019/12/17 15:58:45 At 3:26: Unknown token: 3:26 IDENT var.environment

Erik Osterman (Cloud Posse)about 6 years ago

yup

Erik Osterman (Cloud Posse)about 6 years ago

(due to HCl2)

Erik Osterman (Cloud Posse)about 6 years ago

@Andriy Knysh (Cloud Posse) I don't have access to my terminal ATM

Erik Osterman (Cloud Posse)about 6 years ago

can you check if you can run readme on that project right now?

Erik Osterman (Cloud Posse)about 6 years ago

https://sweetops.slack.com/archives/CRG4CCS03/p1576616261025800

Andriy Knysh (Cloud Posse)about 6 years ago

will check (but I ran it about two weeks ago)

Andriy Knysh (Cloud Posse)about 6 years ago

runs ok

Andriy Knysh (Cloud Posse)about 6 years ago

README and terraform.md generated

Andriy Knysh (Cloud Posse)about 6 years ago

@Igor IDENT error usually happens when you mix different TF versions of TF binary and the module version

Andriy Knysh (Cloud Posse)about 6 years ago

e.g. try to use TF 0.12 version of the module with TF 0.11 binary

Andriy Knysh (Cloud Posse)about 6 years ago

or other way around

Igorabout 6 years ago

Looks like terraform-docs doesn't support HCL2

Igorabout 6 years ago

Hence the error

Igorabout 6 years ago

I don't know enough about awk to troubleshoot why it's failing, but it'd be great to figure it out so others don't get stuck with the same problem

Erik Osterman (Cloud Posse)about 6 years ago

@Andriy Knysh (Cloud Posse) can you confirm that you can reproduce?

Igorabout 6 years ago

I think it's just me

Igorabout 6 years ago

I'll try to find time to debug it on my own

Erik Osterman (Cloud Posse)about 6 years ago

Weird! Something else though I was thinking is we should just run build harness inside of Docker

Erik Osterman (Cloud Posse)about 6 years ago

And eliminate these issues

Erik Osterman (Cloud Posse)about 6 years ago

We already have the container

Erik Osterman (Cloud Posse)about 6 years ago

And already do this in our other pipelines

Igorabout 6 years ago

I am guessing if I run geodesic, it would be more likely to work as well, right?

Andriy Knysh (Cloud Posse)about 6 years ago

yes try in geodesic

Andriy Knysh (Cloud Posse)about 6 years ago

and yes, I tried it many times, works (with TF 0.12 binary and 0.12 version of the module, master branch)

Taylorabout 6 years ago

@Taylor has joined the channel

Roderik van der Veerabout 6 years ago

@Roderik van der Veer has joined the channel

Tarasabout 6 years ago

@Taras has joined the channel

Kostis (Codefresh)about 6 years ago

@Kostis (Codefresh) has joined the channel

Eugeneabout 6 years ago

@Eugene has joined the channel

Tertius Standerabout 6 years ago

@Tertius Stander has joined the channel

curious deviantabout 6 years ago

@curious deviant has joined the channel

Can Kutlu Kinayabout 6 years ago

@Can Kutlu Kinay has joined the channel

deftunixabout 6 years ago

@deftunix has joined the channel

deftunixabout 6 years ago

Hi all, do you have in build-harness any switch to deploy helm charts?

Erik Osterman (Cloud Posse)about 6 years ago

No, or at least not maintained. We use #helmfile for deploying charts

No earlier messagesBrowse by date January 2020