#codefresh - April 2019 | Slack Archive

122 messages

Archive: https://archive.sweetops.com/codefresh/

Mar 2019

April 2019

May 2019

Mark Grubealmost 7 years ago

how many variables are you setting at build time? I usually just use environment variables that are set at run time instead of build time. I add the environment variables through the pod definition. This allows my Dockerfile to be pretty generic

Mark Grubealmost 7 years ago

I see your screenshot that your are setting aws secrets and the port for your container. I don’t think the port matters on your Docker container, so I am assuming its more about your aws secrets. I never was a fan of keeping keys lying around, and I liked having everything controlled through IAM. May I suggest taking a look at https://github.com/uswitch/kiam ? That’s what we use to make pods have specific IAM permissions, and it works great.

Erik Osterman (Cloud Posse)almost 7 years ago

https://github.com/cloudposse/testing.cloudposse.co/pull/22/files

Erik Osterman (Cloud Posse)almost 7 years ago

We should move that “use_codefresh” direnv function into geodesic

Erik Osterman (Cloud Posse)almost 7 years ago

@oscarsullivan_old that’s the default behavior of geodesic

Erik Osterman (Cloud Posse)almost 7 years ago

that’s why you can run

Erik Osterman (Cloud Posse)almost 7 years ago

docker run myco/myinfra | bash

Erik Osterman (Cloud Posse)almost 7 years ago

to install geodesic

Erik Osterman (Cloud Posse)almost 7 years ago

to avoid that, do this:

Erik Osterman (Cloud Posse)almost 7 years ago

https://github.com/cloudposse/testing.cloudposse.co/pull/22/files#diff-fee4a8e1fcec1aac118c7e23c3e707ebR25

Erik Osterman (Cloud Posse)almost 7 years ago


cmd:
      - "-l"
      - "-c"
      - "./tests/run.sh"

Erik Osterman (Cloud Posse)almost 7 years ago

i am not sure about command vs cmd

Erik Osterman (Cloud Posse)almost 7 years ago

https://codefresh.io/docs/docs/codefresh-yaml/steps/freestyle/

Erik Osterman (Cloud Posse)almost 7 years ago

looks like cmd is canonical

Erik Osterman (Cloud Posse)almost 7 years ago

oh, so yea, cmd is the arg passe to the entrypoint

Erik Osterman (Cloud Posse)almost 7 years ago

vs commands is run after the entrypoint

Erik Osterman (Cloud Posse)almost 7 years ago

you can try this:

Erik Osterman (Cloud Posse)almost 7 years ago


cmd: ["-l", "-c", "true"]
commands:
      - "my command"

Erik Osterman (Cloud Posse)almost 7 years ago

hrm… maybe

Erik Osterman (Cloud Posse)almost 7 years ago

😃

Erik Osterman (Cloud Posse)almost 7 years ago

nono

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

in that example, you are passing --version to the entrypoint which is bash

Erik Osterman (Cloud Posse)almost 7 years ago

so you should get the version of bash back

Erik Osterman (Cloud Posse)almost 7 years ago

you could alternatively change the entrypoint

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

study up on ENTRYPOINT vs CMD in docker

Erik Osterman (Cloud Posse)almost 7 years ago

then see what we’re doing in the Dockerfile for geodesic

Erik Osterman (Cloud Posse)almost 7 years ago

yea, the subtle nuances are often misunderstood by Dockerfile authors

Erik Osterman (Cloud Posse)almost 7 years ago

and you see them misused and abused

Erik Osterman (Cloud Posse)almost 7 years ago

I liken it to the system call


int execve(const char *filename, char *const argv[],
                  char *const envp[]);

Erik Osterman (Cloud Posse)almost 7 years ago

ENTRYPOINT ~ filename

Erik Osterman (Cloud Posse)almost 7 years ago

CMD ~ argv

Erik Osterman (Cloud Posse)almost 7 years ago

(this is my mental model, not the docker explanation)

Erik Osterman (Cloud Posse)almost 7 years ago

You could also clone as a step in the codefresh pipeline

Erik Osterman (Cloud Posse)almost 7 years ago

that would be accessible from inside of geodesic

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

the benefit of doing it that way is that you can leverage the git integrations already available in codefresh without putting a secret (e.g. /conf/ssh/deployment-repo-key into git)

rssalmost 7 years ago

Webhooks issue
Apr 3, 09:27 UTC
Resolved - The incident been resolved
Apr 3, 09:24 UTC
Investigating - We are currently investigating an issue where git webhooks don't trigger pipelines.
Running builds manually works as expected

rssalmost 7 years ago

Codefresh availability issue
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability

rssalmost 7 years ago

Codefresh availability issue
Apr 3, 12:46 UTC
Monitoring - Codefresh site is up, we're still monitoring the system
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability

rssalmost 7 years ago

Codefresh availability issue
Apr 3, 13:34 UTC
Update - Google just updated us on an ongoing incident, we're monitoring the issue with google team
Apr 3, 12:46 UTC
Monitoring - Codefresh site is up, we're still monitoring the system
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability

rssalmost 7 years ago

Codefresh availability issue
Apr 3, 14:23 UTC
Resolved - This incident has been resolved.
Apr 3, 14:22 UTC
Update - We are continuing to monitor for any further issues.
Apr 3, 13:34 UTC
Update - Google just updated us on an ongoing incident, we're monitoring the issue with google team
Apr 3, 12:46 UTC
Monitoring - Codefresh site is up, we're still monitoring the system
Apr 3, 12:29 UTC
Investigating - We are currently investigating a problem affecting Codefresh availability

Erik Osterman (Cloud Posse)almost 7 years ago

@Igor Rodionov ☝️

Igor Rodionovalmost 7 years ago

do you have declated env vars
https://github.com/cloudposse/example-app/blob/29f91a718522e4a702d77d172a41ed1f779d42fe/codefresh/pull-request.yaml#L24

Igor Rodionovalmost 7 years ago

@oscarsullivan_old ^

Igor Rodionovalmost 7 years ago

check this env vars

Igor Rodionovalmost 7 years ago

https://github.com/cloudposse/example-app/blob/29f91a718522e4a702d77d172a41ed1f779d42fe/codefresh/pull-request.yaml#L24

Igor Rodionovalmost 7 years ago

https://github.com/cloudposse/build-harness/blob/master/modules/slack/Makefile#L17

Igor Rodionovalmost 7 years ago

https://github.com/cloudposse/build-harness/blob/master/modules/slack/Makefile#L17

Igor Rodionovalmost 7 years ago

To debug it would be useful to see env vars and codefresh.yaml

caseyalmost 7 years ago

@oscarsullivan_old I just setup a pipeline using the codefresh/notify/slack/deploy/webapp target using BUILD_HARNESS_VERSION=0.18.0

caseyalmost 7 years ago

I would check your environment variables and make sure they are set correctly, I was getting similar errors, and it was because of env variables not being set

caseyalmost 7 years ago

does anyone have suggestions for the strategy I should use for deploying to production from codefresh? I feel unsafe giving the ClusterRole codefresh-role permission to deploy to the whole cluster

caseyalmost 7 years ago

Or is that the only way possible?

Erik Osterman (Cloud Posse)almost 7 years ago

We’ve added support for DocumentDB (MongoDB) to our Codefresh Enterprise terraform module: https://github.com/cloudposse/terraform-aws-codefresh-backing-services

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

This makes everything that is absolutely essential to running Codefresh Enterprise (onprem) a fully managed service by AWS.

dustinvbalmost 7 years ago

Awesome! I made this little example app a little more deployable... Still working out the kinks. https://github.com/codefresh-contrib/example-voting-app

dustinvbalmost 7 years ago

Requires a few IPs right now and only works with Cloud LB not sure how to make this more portable using Istio or NGINX as both would require a DNS integration using the LB as is while requiring 2 IPs per running Helm Release is a bit much but this is also meant to demo a few things then toss after playing.

rssalmost 7 years ago

Bitbucket access issue
Apr 11, 11:15 UTC
Investigating - We are currently investigating a problem affecting bitbucket integrations

rssalmost 7 years ago

Build services disruption
Apr 11, 13:11 UTC
Investigating - We are currently investigating this issue

rssalmost 7 years ago

Bitbucket access issue
Apr 11, 14:43 UTC
Resolved - The incident has been resolved
Apr 11, 11:15 UTC
Investigating - We are currently investigating a problem affecting bitbucket integrations

rssalmost 7 years ago

Build services disruption
Apr 11, 14:43 UTC
Resolved - The incident has been resolved
Apr 11, 13:11 UTC
Investigating - We are currently investigating this issue

Erik Osterman (Cloud Posse)almost 7 years ago

https://cloudposse.com/slides/gitops-with-terraform-on-codefresh-webinar/

Erik Osterman (Cloud Posse)almost 7 years ago(edited)

Here are the slides from the webinar today.

Erik Osterman (Cloud Posse)almost 7 years ago

Video will be posted as soon as it's available.

Erik Osterman (Cloud Posse)almost 7 years ago

Here's what a PR looks like: https://github.com/cloudposse/testing.cloudposse.co/pull/75

Erik Osterman (Cloud Posse)almost 7 years ago

Here's the configuration: https://github.com/cloudposse/testing.cloudposse.co/tree/master/codefresh/terraform

rssalmost 7 years ago

Helm repositories issue
Apr 11, 19:04 UTC
Resolved - A fix has been deployed to production and confirmed to have fixed the issue. Helm repositories are fully operational.
Apr 11, 18:49 UTC
Monitoring - The issue has been identified, and the corresponding fix was applied.
Helm repositories are accessible again,
We're closely monitoring the platform to ensure everything is working as expected.
Apr 11, 18:47 UTC
Identified - Currently Helm repositories provided by Codefresh are having issues.
This issue is under...

dustinvbalmost 7 years ago(edited)

@Erik Osterman (Cloud Posse) Do you happen to have an example of running https://github.com/cloudposse/github-commenter as a step in Codefresh pipeline using a Docker image? I've seen in other project snippets of codefresh.yml formatted examples but nothing in this project.

dustinvbalmost 7 years ago

Came up with this really quickly.


  GitHubCommenter:
    title: Add GitHub Comment
    image: cloudposse/github-commenter:latest
    environment:
      - GITHUB_TOKEN=${{GITHUB_TOKEN}} #Must be created see link for info below.
      - GITHUB_OWNER=${{CF_REPO_OWNER}}
      - GITHUB_REPO=${{CF_REPO_NAME}}
      - GITHUB_COMMENT_TYPE=pr
      - GITHUB_PR_ISSUE_NUMBER=${{CF_PULL_REQUEST_NUMBER}}
      - GITHUB_COMMENT="" #Your custom comment goes here

Andriy Knysh (Cloud Posse)almost 7 years ago

this is the latest example that Erik did for the presentation https://github.com/cloudposse/testing.cloudposse.co/blob/osterman-patch-1/codefresh/terraform/pipeline.yml

Andriy Knysh (Cloud Posse)almost 7 years ago

the PR looks like this https://github.com/cloudposse/testing.cloudposse.co/pull/75

dustinvbalmost 7 years ago

Thanks!

Andriy Knysh (Cloud Posse)almost 7 years ago

uses this template file for github-commenter https://github.com/cloudposse/testing.cloudposse.co/blob/osterman-patch-1/codefresh/terraform/pipeline.yml#L59 (update it for your needs)

Andriy Knysh (Cloud Posse)almost 7 years ago

https://github.com/cloudposse/testing.cloudposse.co/blob/osterman-patch-1/codefresh/terraform/comment.txt.gotmpl

Erik Osterman (Cloud Posse)almost 7 years ago

https://m.youtube.com/watch?v=AUOV0XMR_9Q

dustinvbalmost 7 years ago

Is there a how-to on helmfile in Codefresh like an example codefresh.yml with an explanation on the steps?

Erik Osterman (Cloud Posse)almost 7 years ago

Not that well documented

Erik Osterman (Cloud Posse)almost 7 years ago

https://github.com/cloudposse/example-app/tree/master/deploy

Erik Osterman (Cloud Posse)almost 7 years ago

The “ctl” command can be ignored. It just shows how to call Helmfile

Erik Osterman (Cloud Posse)almost 7 years ago

In this example, we support both blue/green and rolling

Erik Osterman (Cloud Posse)almost 7 years ago

Also, this example shows how we use the monochart

Erik Osterman (Cloud Posse)almost 7 years ago

I’d be happy to jump on a call and walk you through it

Mark Grubealmost 7 years ago

How are you guys running stuff like integration tests and system tests? CF can’t directly access internal services that are deployed, so are you guys using some kind of proxy container in the cluster to run these kinds of tests?

Erik Osterman (Cloud Posse)almost 7 years ago

https://github.com/codefresh-io/venona

Erik Osterman (Cloud Posse)almost 7 years ago

This allows you to run it all in your k8s cluster

Erik Osterman (Cloud Posse)almost 7 years ago

we've used this for things like connecting to artifactory or using consul

dustinvbalmost 7 years ago

Yup this runner allows you to talk to the internal pods of the Kubernetes cluster using the container ips if needed.

Mark Grubealmost 7 years ago

this will just show up as a different run time environment on your pipeline?

dustinvbalmost 7 years ago

Add the runner to the cluster and then select the runner for the pipeline and everything happens on your cluster.

dustinvbalmost 7 years ago

Yes

dustinvbalmost 7 years ago

https://codefresh.io/docs/docs/enterprise/behind-the-firewall/

Mark Grubealmost 7 years ago(edited)

Is there a way to split up what runs on CF runtime env and what runs on our infra? I assume that venona still reports everything up to CF UI, right?

dustinvbalmost 7 years ago

Yes all comes back to UI

dustinvbalmost 7 years ago

Each pipeline has a runtime setting. You can use our CLI to call another pipeline from your original as a step in the original and have that child pipeline run on your infra.

Michael Kolbalmost 7 years ago

Has anyone been using cf_export stating the name of an existing environment variable? I have tried to use it, but the variable output is getting altered.
FreeStye Bash Code

 - FEATURE=$(echo ${{CF_BRANCH}} | cut -c 9- )
      - echo $FEATURE
      - cf_export $FEATURE

Output

TI-595                                                                                                                                                  
Exporting TI-595=-595

I am seeing the cf_export command parse the static environment variable value vs. storing the defined value with the assignment cf_export $FEATURE.

Andriy Knysh (Cloud Posse)almost 7 years ago

@Michael Kolb here’s a pipeline step example to setup ENV vars

Andriy Knysh (Cloud Posse)almost 7 years ago

  env:
    title: Setup Environment
    stage: Init
    fail_fast: true
    image: ${{build_image}}
    working_directory: &cwd ${{main_clone}}/${{PROJECT}}
    commands:
      - cf_export BUILD_HARNESS_VERSION=0.18.0

      # Github Commenter
      - cf_export GITHUB_OWNER=${{CF_REPO_OWNER}}
      - cf_export GITHUB_REPO=${{CF_REPO_NAME}}
      - cf_export GITHUB_COMMENT_TYPE=pr
      - cf_export GITHUB_PR_ISSUE_NUMBER=${{CF_PULL_REQUEST_NUMBER}}
      - cf_export GITHUB_COMMENT_FORMAT_FILE=${{CF_VOLUME_PATH}}/${{CF_REPO_NAME}}/codefresh/terraform/comment.txt.gotmpl

Andriy Knysh (Cloud Posse)almost 7 years ago

it exports Codefresh env vars and also those defined in Codefresh UI (Global config section and ENV vars section in each pipeline)

Andriy Knysh (Cloud Posse)almost 7 years ago

you mean the example does not use the ${{...}} syntax?

Michael Kolbalmost 7 years ago

So, in the documentation it says that you can state an existing variable
state the name of an existing environment variable (like EXISTING_VAR)
https://codefresh.io/docs/docs/codefresh-yaml/variables/#using-cf_export-command

Michael Kolbalmost 7 years ago

However, this hasn’t stored that variable assignment

Michael Kolbalmost 7 years ago

and the displayed use case does not show assignment in the code

Michael Kolbalmost 7 years ago

ersion: '1.0'
steps:
  freestyle-step-1:
    description: Freestyle step..
    title: Free styling
    image: alpine:latest
    commands:
      - cf_export VAR1=VALUE1 VAR2=VALUE2 EXISTING_VAR

  freestyle-step-2:
    description: Freestyle step..
    title: Free styling 2
    image: ${{VAR1}}
    commands:
      - echo $VAR2
      - curl http://$EXISTING_VAR/index.php

Michael Kolbalmost 7 years ago(edited)

is the documentation referencing a pipeline variable defined in the pipeline vs. in the pipeline yaml?

dustinvbalmost 7 years ago

Yes pipeline variables are in the pipeline SPEC file. If you need variables for the pipeline to be set during pipeline execution then you'd use cf_export to have them pop out to pipeline.

dustinvbalmost 7 years ago

The existing variable section is mentioning step-2 in the YAML above.

dustinvbalmost 7 years ago

You do not need to specify a variable name before the cf_export

Michael Kolbalmost 7 years ago(edited)

thanks, the documentation page could use more examples of cf_export variable assignment like the example that you provided - cf_export GITHUB_OWNER=${{CF_REPO_OWNER}}

Michael Kolbalmost 7 years ago

they display assignment as static references

Michael Kolbalmost 7 years ago

and the reference EXISTING_VAR is misleading if this is pulling from the pipeline variable definition vs. an existing variable

Michael Kolbalmost 7 years ago

documentation clarification would help, since I have had to run several builds trying to figure out how the cf_export process handles environment variables

Michael Kolbalmost 7 years ago

is there any documentation on the codefresh/kube-helm image that provides connection to Kubernetes & Tiller?

dustinvbalmost 7 years ago

Are you talking about the image used in our UI steps https://github.com/codefresh-contrib/images/tree/master/kube-helm or Helm step https://github.com/codefresh-contrib/cfstep-helm

Michael Kolbalmost 7 years ago

the docker hub release

Michael Kolbalmost 7 years ago(edited)

i found the command that i need kubectl config use-context <<cluster name>>

Michael Kolbalmost 7 years ago

the container had all the tools installed. Unfortunately, there was not documentation on the docker hub site.

dustinvbalmost 7 years ago

I would recommend trying to set the working_directory: ${{main_clone}} for the step to ensure you're in the clone repository directory.

rssalmost 7 years ago

Docker incident: Docker Hub elevated errors
Apr 25, 17:33 UTC
Investigating - Docker has reported an incident affecting Docker Hub Web and Docker Hub Automated Builds components.
It shouldn't affect normal pulling and pushing operations.
We will keep monitoring this.

More information here: https://status.docker.com/

rssalmost 7 years ago

Docker incident: Docker Hub elevated errors
Apr 25, 18:10 UTC
Resolved - Docker has reported this incident as Resolved.
"[Resolved] The issue has been resolved. The DB is backup again."

More information here: https://status.docker.com/pages/incident/533c6539221ae15e3f000031/5cc1ed78790d0e1ca1c8fcd4
Apr 25, 17:33 UTC
Investigating - Docker has reported an incident affecting Docker Hub Web and Docker Hub Automated Builds components.
It shouldn't affect normal pulling and pushing operations.
We will keep monitoring this.

More...

Michael Kolbalmost 7 years ago(edited)

Does Codefresh have API that accepts commands to start an existing pipeline? I have an integration test that will take hours and would like the option to start another pipeline once the integration tests complete.

dustinvbalmost 7 years ago

https://codefresh-io.github.io/cli/pipelines/run-pipeline/

Usage from a step in pipeline.

https://github.com/codefresh-contrib/example-voting-app/blob/master/.codefresh/codefresh-cd.yml#L72-L76

March 2019 Browse by date May 2019